Intelligent detection system for forklift collision based on multi-source state data and neural network model

Through the intelligent detection system of multi-source state data and neural network model, combined with multiple sensors and deep learning algorithms, the problem of insufficient robustness of traditional forklift collision detection systems is solved, and high accuracy and stability in complex environments are achieved Test.

CN118797487BActive Publication Date: 2024-11-15AIDONG SUPER AI
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411268042.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-11
Publication Date
2024-11-15
Estimated Expiration
2044-09-11

AI Technical Summary

Technical Problem

Traditional forklift collision detection systems rely on single sensor data, are not robust enough, and are prone to misjudgment and misjudgment due to changes in environmental noise and working conditions. They also use simple threshold judgment methods to achieve poor results in complex environments.

Method used

An intelligent detection system adopts multi-source state data and neural network model, integrates motion state sensors and vibration state sensors, and combines convolutional neural networks and recursive neural networks through time-frequency analysis, dynamic parameter analysis, data fusion and information entropy analysis models, and combines convolutional neural networks and recursive neural networks. Perform comprehensive judgment and feature extraction to replace simple threshold judgment.

Benefits of technology

It improves the accuracy and robustness of forklift collision detection, reduces misjudgment and misjudgment, can accurately identify collision events in complex environments, and enhances the system's environmental adaptability and stability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118797487B_ABST
    Figure CN118797487B_ABST
Patent Text Reader

Abstract

The present application provides an intelligent detection system for forklift collision based on multi-source state data and a neural network model, which includes: a motion state sensor detects the motion state of the forklift to obtain motion state data; a vibration state sensor detects the vibration state of the forklift to obtain vibration state data; an on-board data acquisition terminal collects motion state data and vibration state data to obtain motion state sampling data and vibration state sampling data; a data analysis server is deployed with a time-frequency analysis model, a dynamic parameter analysis model, a data fusion model, an information entropy analysis model, and a collision detection model to determine whether the forklift has collided based on the motion state sampling data and the vibration state sampling data, thereby improving the accuracy of detection, etc.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of safety technology, and in particular to an intelligent detection system for forklift collision based on multi-source status data and a neural network model. Background Art

[0002] With the rapid development of the logistics industry, forklifts, as important material handling tools, have been widely used in warehouses, factories and other places. However, the safety hazards in the operation of forklifts, especially collision accidents, have always been a problem that needs to be solved urgently in the industry. Traditional forklift collision detection systems mostly rely on simple threshold judgment methods. This method often does not work well in complex and changeable working environments and is easily affected by environmental noise and changes in working conditions, resulting in frequent misjudgments and missed judgments.

[0003] Specifically, for example, traditional systems usually rely only on a single type of sensor data (such as accelerometer) for collision detection, ignoring the comprehensive use of other motion states (such as speed, angular velocity) and vibration state information, resulting in insufficient robustness of detection. In a complex and changing operating environment, a single data source is difficult to fully reflect the actual movement of the forklift, increasing the risk of misjudgment and missed judgment. Alternatively, existing collision detection systems mostly use a preset fixed threshold to determine whether a collision has occurred. This method is prone to misjudging normal vibrations as collisions, or ignoring actual collision events when the ambient noise is large or the working conditions change, thereby reducing the accuracy of detection. Summary of the invention

[0004] The purpose of this application is to propose an intelligent detection system for forklift collision based on multi-source state data and a neural network model, so as to solve or alleviate the technical problems existing in the prior art.

[0005] The embodiment of the present application provides an intelligent detection system for forklift collision based on multi-source state data and a neural network model, which includes:

[0006] A motion state sensor is used to detect the motion state of the forklift and obtain motion state data, wherein the motion state data includes driving speed data, acceleration data and angular velocity data;

[0007] A vibration state sensor, used for detecting the vibration state of the forklift to obtain vibration state data;

[0008] The vehicle-mounted data acquisition terminal is used to collect the motion state data and the vibration state data to obtain motion state sampling data and vibration state sampling data, wherein the motion state sampling data includes driving speed sampling data, acceleration sampling data and angular velocity sampling data;

[0009] A data analysis server is deployed with a time-frequency analysis model, a dynamic parameter analysis model, a data fusion model, an information entropy analysis model, and a collision detection model to determine whether the forklift has collided based on the motion state sampling data and the vibration state sampling data.

[0010] In the intelligent detection system for forklift collision based on multi-source state data and neural network model of the present application, the system integrates multiple sensors, including motion state sensors (used to detect driving speed, acceleration, angular velocity, etc.) and vibration state sensors. This multi-source data collection method enables the system to obtain more comprehensive and richer forklift motion information. Through the fusion processing of multi-sensor data, the system can comprehensively consider the different motion states and vibration characteristics of the forklift to improve the accuracy and robustness of collision detection. Compared with single sensor data, the fusion of multi-source data can better cope with complex and changeable working environments and reduce misjudgments and missed judgments caused by insufficient single data sources. In the data analysis stage, not only the vibration data is analyzed, but also the motion state data (such as speed, acceleration, angular velocity, etc.) is combined for comprehensive judgment. This comprehensive analysis method can more accurately reflect the actual motion state of the forklift and improve the accuracy of collision detection.

[0011] Furthermore, the system of the present invention adopts an intelligent collision detection model based on machine learning, replacing the traditional simple threshold judgment method. The model trains a large amount of historical data through a deep learning algorithm, and can automatically learn and extract the complex features of forklift collision events. Compared with simple threshold judgment, the intelligent collision detection model has stronger environmental adaptability and robustness. It can accurately identify collision events in complex and changing environments, and reduce misjudgments and missed judgments caused by environmental noise or changes in working conditions. Finally, the convolutional neural network (CNN) model in the system is responsible for extracting the spatial features of the vibration signal, while the recursive neural network (RNN) model is used to extract time series features. The combination of these two models enables the system to extract richer feature information from complex sensor data, providing a more accurate basis for collision detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Hereinafter, some specific embodiments of the present application will be described in detail in an exemplary and non-limiting manner with reference to the accompanying drawings. The same reference numerals in the accompanying drawings indicate the same or similar components or parts. It should be understood by those skilled in the art that these drawings are not necessarily drawn to scale. In the accompanying drawings:

[0013] Figure 1 This is a flow chart of an intelligent detection system for forklift collision based on multi-source status data and a neural network model according to an embodiment of the present application. DETAILED DESCRIPTION

[0014] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the embodiments of the present application should fall within the scope of protection of the embodiments of the present application.

[0015] Figure 1 This is a schematic diagram of the structure of an intelligent detection system for forklift collision based on multi-source state data and a neural network model according to an embodiment of the present application. Figure 1 As shown, it includes:

[0016] The motion state sensor 1 is used to detect the motion state of the forklift 2 and obtain motion state data, wherein the motion state data includes driving speed data, acceleration data and angular velocity data;

[0017] A vibration state sensor 3, used for detecting the vibration state of the forklift 2 to obtain vibration state data;

[0018] The vehicle-mounted data acquisition terminal 4 is used to collect the motion state data and the vibration state data to obtain motion state sampling data and vibration state sampling data, wherein the motion state sampling data includes driving speed sampling data, acceleration sampling data and angular velocity sampling data;

[0019] The data analysis server 5 has a collision detection model deployed thereon, which is used to determine whether the forklift 2 has a collision according to the motion state sampling data and the vibration state sampling data.

[0020] Optionally, a remote communication terminal 6 is installed on the forklift 2 to transmit the motion state sampling data and the vibration state sampling data to the data analysis server 5. To this end, the motion state sensor 1 can be directly integrated on the remote communication terminal 6 to improve the integration.

[0021] Optionally, the vehicle-mounted data acquisition terminal 4 is configured with a data interface, and the data interface is connected to the motion state sensor 1 and the vibration state sensor 3 to collect the motion state data and the vibration state data.

[0022] To this end, the optional data interface on the vehicle-mounted data acquisition terminal 4 has the following technical advantages:

[0023] (1) The vehicle-mounted data acquisition terminal 4 has built-in motion state sensor 1 and vibration state sensor 3, and supports configuration of data interfaces, so that the system can flexibly connect different types of motion state sensors 1 and vibration state sensors 3. This design not only meets the current needs of built-in sensors, but also reserves interfaces for sensors that may be added in the future. With the development of technology and changes in detection requirements, the system can easily expand the types and quantities of sensors to adapt to a wider range of detection needs, thereby improving the overall flexibility and scalability of the system.

[0024] (2) Connecting sensors through standardized data interfaces can simplify system wiring and installation processes. Compared with direct welding or other complex connection methods, data interfaces provide a more convenient and reliable connection method. This not only reduces installation difficulty and cost, but also improves installation efficiency and reduces the possibility of human error.

[0025] (3) During the operation of the equipment, the sensor may need to be replaced due to damage or aging. After configuring the data interface, the replacement of the sensor becomes simpler and faster. The maintenance personnel do not need to dismantle the entire data acquisition terminal, but only need to disconnect the interface and replace the new sensor, which greatly reduces the difficulty and time cost of maintenance.

[0026] (4) Data interfaces usually have high electrical performance and stability, and can effectively resist electromagnetic interference and signal attenuation. Connecting sensors through data interfaces can improve the reliability and accuracy of signal transmission and reduce misjudgments and missed judgments caused by signal transmission problems. At the same time, data interfaces also support hot-swap functions, which allows sensors to be replaced without interrupting system operation, further improving the stability and reliability of the system.

[0027] Optionally, the data interface is an analog quantity interface, and the motion state data and the vibration state data are analog quantities; when the data interface is a digital quantity interface, the motion state data and the vibration state data are digital quantities.

[0028] Combined with the technical solution, the optional data interface type (analog interface or digital interface) has the following technical benefits:

[0029] (1) When the motion state data and vibration state data exist in the form of analog quantities, the analog quantity interface can directly obtain these continuously changing signals. The analog quantity signal can more accurately reflect the slight changes of the forklift 2 during the movement process, such as slight fluctuations in acceleration or slight differences in vibration. This is particularly important for application scenarios that require high-precision collision detection, and can improve the accuracy and reliability of collision detection.

[0030] (2) Analog interfaces have a wide range of applications in the field of industrial automation, and many traditional sensors and instruments support analog output. Therefore, the use of analog interfaces can seamlessly connect with these existing devices without the need for additional conversion equipment, reducing the cost and complexity of system integration.

[0031] (3) Analog signals do not undergo digital quantization during transmission and processing, thus avoiding errors caused by quantization step length. This is particularly important for collision detection systems that need to accurately capture dynamic changes and ensure the authenticity and integrity of data.

[0032] (4) Digital signals have higher anti-interference ability and can be transmitted stably in complex electromagnetic environments. Compared with analog signals, digital signals are less susceptible to noise and interference, and can ensure the accuracy and reliability of data. During the operation of forklift 2, various electromagnetic interference sources may be encountered. The use of digital interfaces can improve the stability and reliability of the system.

[0033] (5) Digital signals can be transmitted remotely through standard communication protocols, supporting long-distance, high-speed data exchange. This makes it easier for the system to achieve centralized data processing and cloud-based analysis, improving the efficiency and accuracy of data processing. At the same time, digital signals are also easy to integrate with various digital devices and systems, expanding the system's functions and application scope.

[0034] (6) When transmitting the same number of data points, the bandwidth required for digital signals is usually smaller than that for analog signals. This is because digital signals can remove redundant information and improve data transmission efficiency through encoding and compression. In application scenarios with limited bandwidth or cost sensitivity, the use of digital interfaces can reduce the communication cost and maintenance difficulty of the system.

[0035] Optionally, the data analysis server 5 is deployed with a time-frequency analysis model, a dynamic parameter analysis model, a data fusion model, an information entropy analysis model, and a collision detection model, wherein:

[0036] The time-frequency analysis model is used to transform the vibration state sampling data from the time domain to the frequency domain to obtain a vibration characteristic value sequence of the vibration state sampling data in the frequency domain;

[0037] The dynamic parameter analysis model is used to calculate the driving speed gradient sequence, the acceleration change rate sequence, and the angular velocity change rate sequence according to the driving speed sampling data, the acceleration sampling data, and the angular velocity sampling data respectively;

[0038] The data fusion model is used to fuse the vibration characteristic value sequence, the driving speed gradient sequence, the acceleration change rate sequence, and the angular velocity change rate sequence to obtain fused data;

[0039] The information entropy analysis model is used to perform information entropy analysis on the fused data sequence to evaluate the information gain of the fused data sequence and determine whether the information gain reaches a preset information gain threshold. If so, the collision detection model is triggered to determine whether the forklift 2 has a collision based on the fused data sequence; otherwise, the fused data sequence is amplified so that the information gain of the amplified fused data sequence reaches the preset information gain threshold.

[0040] To this end, the above-mentioned solution based on the time-frequency analysis model, dynamic parameter analysis model, data fusion model and information entropy analysis model deployed on the data analysis server 5 has the following technical advantages:

[0041] (1) The time-frequency analysis model and the dynamic parameter analysis model are used to extract feature values ​​from the two dimensions of vibration characteristics and dynamic motion state, respectively, and then the data fusion model is used to fuse them to generate fused data containing richer information. This fusion of multi-dimensional features enables the system to more comprehensively reflect the actual operating state of the forklift 2, thereby improving the accuracy of collision detection.

[0042] (2) The information entropy analysis model performs information entropy analysis on the fused data sequence and evaluates its information gain. Only when the information gain reaches the preset threshold will the collision detection model be triggered to make the final judgment. This mechanism ensures that collision detection is only performed when the data quality is high enough, further improving the accuracy of detection.

[0043] (3) The data provided by the time-frequency analysis model and the dynamic parameter analysis model verify and complement each other during the fusion process, which improves the fault tolerance of the system. Even if the data in one dimension is abnormal, the system can correct it through the data in other dimensions to ensure the accuracy and reliability of the overall data.

[0044] (4) The information entropy analysis model evaluates the information gain of the fused data sequence and amplifies the fused data sequence when necessary to ensure data quality. This adaptive adjustment mechanism enables the system to maintain stable detection performance under different working conditions and environmental noise, thereby enhancing the robustness of the system.

[0045] (5) The various models on the data analysis server 5 work together to form an efficient data processing process. The time-frequency transformation, dynamic parameter calculation, data fusion and information entropy analysis are closely linked, reducing the delay of data processing. This efficient processing process enables the system to quickly respond to changes in the operating status of the forklift 2 and improve the real-time performance of the detection.

[0046] (6) The information entropy analysis model determines whether to trigger the collision detection model by evaluating the information gain of the fused data sequence. This precise triggering mechanism avoids unnecessary detection when the data quality is poor, thereby reducing the false alarm rate. At the same time, the fusion of multi-dimensional features improves the accuracy of detection and reduces the false alarm rate.

[0047] (7) When the information gain of the fused data sequence does not reach the preset threshold, the information entropy analysis model triggers data amplification processing to increase the information gain. This intelligent data processing strategy ensures that the system can fully utilize the available data resources before performing collision detection, avoiding detection failure or resource waste due to insufficient data quality.

[0048] Optionally, the data fusion model is used to perform the following steps to fuse the vibration characteristic value sequence, the driving speed gradient sequence, the acceleration change rate sequence, and the angular velocity change rate sequence to obtain a fused data sequence:

[0049] Performing time stamp alignment on the vibration characteristic value sequence, the driving speed gradient sequence, the acceleration change rate sequence, and the angular velocity change rate sequence to obtain a synchronized vibration characteristic value sequence, driving speed gradient sequence, acceleration change rate sequence, and angular velocity change rate sequence;

[0050] Recursively filtering the synchronized vibration characteristic value sequence, driving speed gradient sequence, acceleration change rate sequence, and angular speed change rate sequence to obtain calibrated vibration characteristic value sequence, driving speed gradient sequence, acceleration change rate sequence, and angular speed change rate sequence;

[0051] The calibrated vibration characteristic value sequence, driving speed gradient sequence, acceleration change rate sequence, and angular velocity change rate sequence are fused to obtain a fused data sequence.

[0052] To this end, the specific steps performed based on the data fusion model have the following technical benefits:

[0053] (1) The data fusion model first aligns the timestamps of the vibration eigenvalue sequence, driving speed gradient sequence, acceleration rate of change sequence, and angular velocity rate of change sequence to ensure that data from different sources remain synchronized in the time dimension. This is the premise of data fusion, avoiding fusion errors caused by time asynchrony and improving the accuracy and reliability of the fused data sequence.

[0054] (2) By recursively filtering the synchronized sequence, the noise and outliers in the data can be filtered out to obtain a calibrated sequence. Recursive filtering can not only effectively reduce random errors in the data, but also retain the effective information of the data, improve the quality of the data, and provide a better foundation for subsequent fusion and analysis.

[0055] (3) After timestamp alignment and recursive filtering, the data is fused on the basis of maintaining synchronization and high quality, which can obtain a more comprehensive and accurate fused data sequence. This fusion not only makes full use of the complementarity of multi-source data, but also improves the fusion effect through reasonable data processing methods, so that the fused data can better reflect the actual operating status of forklift 2.

[0056] (4) The fused data sequence contains multi-dimensional features such as vibration gradient change, driving speed gradient, acceleration change rate, angular velocity change rate, etc. These features complement and verify each other, providing more comprehensive and accurate information for the collision detection model. This fusion of multi-dimensional features enables the collision detection model to more accurately determine whether forklift 2 has collided, thereby improving the accuracy and robustness of detection.

[0057] (5) Through the processing of the data fusion model, a high-quality fused data sequence is obtained. This data is not only highly accurate, but also contains rich information about the operating status of forklift 2. Collision detection based on this data can more accurately identify real collision events, reduce false alarms and missed alarms, and improve the reliability and stability of the system.

[0058] (5) As an important part of the data processing flow, the data fusion model ensures the response speed of the entire system with its efficient processing speed. By performing operations such as timestamp alignment, recursive filtering and fusion on multi-source data, the system can quickly obtain high-quality fused data sequences and perform subsequent operations such as collision detection based on these data, thereby improving the overall performance of the system.

[0059] Optionally, the synchronized vibration eigenvalue sequence, driving speed gradient sequence, acceleration rate of change sequence, and angular velocity rate of change sequence are respectively mapped to state estimates according to time steps, so as to recursively filter the state estimates based on the following formula to obtain updated state estimates, so as to obtain calibrated vibration eigenvalue sequences, driving speed gradient sequences, acceleration rate of change sequences, and angular velocity rate of change sequences, respectively:

[0060] , used to estimate the state of the previous time step and the system control input To predict the state of the current time step ,in, AIt is a state transfer matrix, which is used to describe the trend estimation of the vibration value, driving speed, acceleration, and angular velocity of the forklift over time when a collision occurs. B is the control input matrix, which is used to describe the influence weight of the control input on the vibration value, driving speed, acceleration and angular velocity of the forklift when a collision occurs;

[0061] , used to calculate the covariance matrix of the predicted state , which measures the uncertainty of the predicted state, where is the state estimate covariance matrix of the previous time step, is the covariance matrix of the process noise;

[0062] , the Kalman gain is calculated based on the state estimate covariance matrix of the previous time step , which determines the weights of the predicted state and the observed value when updating the state estimate, is the observation matrix, To observe the noise covariance matrix, these two matrices are estimated by experimental determination;

[0063] , used to calculate the observed value based on the current time step To update the state estimate for the next time step , combined with the state of the current time step and observations , where the Kalman gain Controlled the weight of these two;

[0064] , used to update the covariance matrix of the state estimate , which reflects the uncertainty of the updated state estimate.

[0065] To this end, the above-mentioned recursive filtering process is performed on the synchronized vibration characteristic value sequence, driving speed gradient sequence, acceleration change rate sequence, and angular velocity change rate sequence to obtain the calibrated driving speed gradient sequence, acceleration change rate sequence, and angular velocity change rate sequence, which has the following technical advantages:

[0066] (1) A prediction-update mechanism is established to effectively filter out noise and outliers in the data. During the operation of forklift 2, due to the complex and changeable environment, the sensor data is easily affected by various external interferences. Through recursive filtering, the impact of these interferences on data quality can be significantly reduced, and the accuracy and reliability of the data can be improved.

[0067] (2) It not only filters out noise, but also maintains a smooth transition of the data, avoiding sudden peaks or valleys in the data. This is particularly important for subsequent data analysis and collision detection, as sudden data points may lead to misjudgment.

[0068] (3) It has dynamic adjustment capability and can adaptively adjust the filter gain according to the estimation error of the previous moment and the observed data of the current moment, thereby dynamically tracking the changing characteristics of the data. This adaptive characteristic makes recursive filtering perform well in processing non-stationary signals, and is particularly suitable for dynamically changing systems such as forklifts 2.

[0069] (4) In the data fusion process, the data processed by recursive filtering has higher quality and can be better fused with other sensor data. This helps to improve the accuracy and reliability of the fused data sequence and provide a more solid foundation for subsequent collision detection.

[0070] (5) Through recursive filtering, the time for data preprocessing can be reduced and the overall processing speed of the system can be improved. At the same time, since the quality of the filtered data is higher, the false alarm rate and missed alarm rate of the collision detection model will also be reduced accordingly, thereby improving the stability and reliability of the system.

[0071] (6) The introduction of recursive filtering enhances the system’s resistance to external interference, allowing the system to maintain stable performance under different working conditions and environmental conditions. This is especially important for equipment such as forklifts that work in complex and changing environments.

[0072] Optionally, the fused vibration characteristic value sequence, driving speed gradient sequence, acceleration change rate sequence, and angular velocity change rate sequence of the calibrated vibration characteristic value sequence to obtain a fused data sequence includes:

[0073] Based on the prior probability distribution and conditional probability distribution of the motion state of the forklift 2, according to the calibrated vibration characteristic value sequence, driving speed gradient sequence, acceleration change rate sequence, and angular velocity change rate sequence, the vibration posterior probability distribution data, the driving speed posterior probability distribution data, the acceleration posterior probability distribution data, and the angular velocity posterior probability distribution data of the forklift 2 are determined;

[0074] The vibration posterior probability distribution data of the forklift 2, the driving speed posterior probability distribution data, the acceleration posterior probability distribution data of the forklift 2, and the angular velocity posterior probability distribution data are fused to obtain a fused data sequence. Here, for example, the fusion is performed by a Bayesian algorithm. Of course, in this application, it is not limited to the Bayesian fusion method, and other fusion algorithms can also be selected according to specific indicator requirements.

[0075] Here, for example, a particle filter can be used to track the motion state of a forklift. By generating a set of particles to represent the possible states in the state space, and updating the weights and positions of the particles according to the observed data, the posterior probability distribution of the state is finally obtained. For example, the specific process may include the following:

[0076] 1. Particle initialization

[0077] Prior probability distribution: At the beginning of the particle filter, a set of particles needs to be generated to approximate the initial probability distribution of the forklift's motion state, that is, the prior probability distribution. This is usually based on the known initial state of the forklift, historical data, or expert knowledge. For example, if it is known that the forklift is stationary at the beginning, then most of the particles can be initialized as points near the stationary state.

[0078] Particle generation: According to the prior probability distribution, a certain number of particles are randomly generated in the state space. Each particle represents a possible state of the forklift (such as vibration value, position, speed, acceleration, etc.).

[0079] 2. Observation data acquisition

[0080] The calibrated vibration characteristic value sequence, driving speed gradient sequence, acceleration rate of change sequence and angular velocity rate of change sequence are obtained as observation data. These data are collected in real time by sensors on the forklift and processed after calibration.

[0081] 3. Particle weight update

[0082] Conditional probability distribution: The observed data provides new information about the current state of the forklift, which is quantified by the conditional probability distribution (i.e., likelihood function). In particle filtering, the likelihood function is used to evaluate the degree of match between the state represented by each particle and the observed data.

[0083] Weight calculation: For each particle, the likelihood (i.e., weight) of the particle matching the observed data is calculated based on its corresponding state and observed data. The weight usually reflects the similarity between the state represented by the particle and the true state.

[0084] Weight normalization: Normalize the weights of all particles to ensure that the sum of their weights is 1, thus maintaining the properties of the probability distribution.

[0085] 4. Particle Resampling

[0086] Since the weights of some particles may become very small after many iterations, resulting in these particles contributing little to the state estimation, particle resampling is required to increase the number of high-weight particles and reduce or eliminate low-weight particles.

[0087] During the resampling process, particles are randomly selected for replication or discard according to their weights to form a new particle set. The new particle set will more concentratedly represent the high probability area of ​​the forklift state.

[0088] 5. Particle state update

[0089] After resampling, the particle’s state is usually further updated based on the forklift’s motion model and the observations. This can be done by combining the particle’s state with the observations and using some form of prediction-update algorithm (such as the prediction step of a Kalman filter).

[0090] 6. State Estimation

[0091] Use the updated set of particles to estimate the current state of the forklift. This is usually done by computing a weighted average state of all particles, where particles with higher weights contribute more to the final estimate.

[0092] Through the above steps, the particle filter can combine the prior probability distribution and conditional probability distribution of the forklift's motion state (through observation data), as well as the calibrated sensor data, to track and estimate the forklift's motion state in real time.

[0093] Therefore, a method for obtaining a fused data sequence by calculating the posterior probability based on the prior probability and the conditional probability to fuse the calibrated vibration characteristic value sequence, the driving speed gradient sequence, the acceleration change rate sequence, and the angular velocity change rate sequence has the following technical advantages:

[0094] (1) Integrating information from multiple data sources, by considering prior probabilities and conditional probabilities, can more accurately reflect the inherent relationships and uncertainties of each data source, thereby generating more reasonable fused data. This method avoids the bias that may be introduced by simple averaging or weighted averaging, and improves the accuracy and reliability of data fusion.

[0095] (2) In the complex and ever-changing operating environment of forklifts, sensor data is often affected by various noises and interferences. The above reasoning can resist these noises and interferences to a certain extent and improve the robustness of the system by considering the statistical characteristics of the data. Even if a certain sensor data is biased or abnormal, these deviations can be corrected by integrating other sensor information to ensure the accuracy of the fused data sequence.

[0096] (3) The obtained posterior probability distribution data not only contains the uncertainty information of the data, but also can reflect the consistency between the data to a certain extent. During the fusion process, if there is conflict or inconsistency between the information of different data sources, these conflicts can be coordinated by adjusting the weights and probability distribution to generate a more consistent and reliable fusion data sequence.

[0097] (4) The fused data sequence will serve as an important basis for subsequent collision detection and risk assessment. The fused data sequence obtained through the above reasoning has higher accuracy and reliability, and can provide more accurate input information for the system, thereby improving the accuracy of collision detection and risk assessment. This is of great significance for ensuring the safety and stability of forklift 2 operation.

[0098] (5) The above reasoning has the ability to be dynamically updated, that is, as new data arrives, the prior probability and conditional probability can be continuously updated to obtain a more accurate posterior probability distribution. This dynamic update capability enables the system to adapt to the changing operating environment of forklift 2 and maintain high detection performance and stability.

[0099] Optionally, the collision detection model determines whether the forklift 2 has collided based on the fused data sequence, including: the collision detection model extracts features from the fused data sequence to obtain spatial features and temporal features, and fuses them to obtain fused features, so as to determine whether the forklift 2 has collided by mapping the fused features to the constructed category labels.

[0100] Therefore, the collision detection model determines whether the forklift 2 has collided based on the fused data. This process is performed by extracting features, fusing spatial features and temporal features, and mapping these features to category labels. This has the following technical benefits:

[0101] (1) The feature extraction step can extract information that is crucial for collision detection from the fused data, including spatial features and temporal features. Spatial features reflect the motion state of forklift 2 in different directions, while temporal features capture the changing trend of these states over time. By fusing these two features, the model can more comprehensively understand the motion pattern of forklift 2, thereby improving the accuracy of collision detection.

[0102] (2) The fusion of spatial features and temporal features makes the model no longer rely on single-dimensional data, but analyzes the motion state of forklift 2 from multiple angles. This multi-dimensional information processing method enhances the model's resistance to noise and outliers. Even in a complex and changing working environment, the model can maintain stable performance and improve the robustness of the system.

[0103] (3) By mapping the fused features to the constructed category labels, the model can distinguish different collision types (such as minor collision, severe collision, etc.) and non-collision events (such as normal driving, emergency braking, etc.). This classification capability enables the system to accurately judge the motion state of forklift 2 in complex scenarios and provide operators with more accurate safety warnings and operation guidance.

[0104] Optionally, the collision detection model includes a convolutional neural network model, a recursive neural network model, a fusion module, and a classifier;

[0105] The convolutional neural network model includes an input layer, a multi-layer convolution and a pooling layer;

[0106] Converting the fused data sequence into an input vector of the multi-layer convolution based on the input layer;

[0107] Performing a convolution operation on the input vector based on the multi-layer convolution to extract local spatial features in the vibration state data;

[0108] Based on the pooling layer, the local spatial features are subjected to dimensionality reduction processing to obtain spatial features;

[0109] The recurrent neural network model includes a serialization module and a gated recurrent unit;

[0110] Based on the serialization module, vectorization processing is performed on the fused data sequence to obtain a time series feature vector;

[0111] Based on the gated recurrent unit, feature extraction is performed on the time series feature vector to capture the time series features in the motion state sampling data;

[0112] Based on the fusion module, the spatial features and the temporal features are fused to obtain fusion features;

[0113] Based on the classifier, the fusion feature is mapped to a predefined category label to determine whether the forklift 2 has a collision.

[0114] To this end, the combined use of convolutional neural networks (CNN) and recurrent neural networks (RNN, especially gated recurrent units (GRU)) in the above collision detection model, as well as the design of fusion modules and classifiers, has the following technical benefits:

[0115] 1. Efficient spatial feature extraction (CNN)

[0116] Local spatial feature extraction: The convolutional neural network can effectively extract local spatial features from the fused data through multi-layer convolution operations. These features represent the specific vibration patterns generated by the forklift 2 during a collision and are crucial for identifying collision events.

[0117] Dimensionality reduction: The pooling layer reduces the dimensionality of local spatial features, reduces the redundancy of data, extracts more critical and representative features, and improves the efficiency and accuracy of subsequent processing.

[0118] 2. Powerful temporal feature capture (RNN-GRU)

[0119] Time series feature extraction: Recurrent neural networks, especially gated recurrent units (GRUs), can capture long-term dependencies in time series data. In forklift 2 collision detection, the changing trend of motion state data (such as driving speed, acceleration, angular velocity) is also important for judging collision events. GRU can effectively extract the time series features of these data through its internal mechanism.

[0120] Flexibility: Compared with traditional RNN, GRU simplifies the model structure, improves training efficiency and performance, and reduces the risk of gradient disappearance or gradient explosion by introducing update gate and reset gate.

[0121] 3. Feature fusion and classification decision

[0122] Feature fusion: The fusion module fuses the spatial features extracted by CNN and the temporal features extracted by RNN-GRU to form a more comprehensive fusion feature. This fusion feature contains both the spatial information of the vibration state and the temporal information of the motion state, which helps to more accurately determine whether the forklift 2 has collided.

[0123] Classification decision: The classifier maps the fusion features to pre-defined category labels (such as "collision" and "non-collision"), achieving rapid and accurate judgment of forklift collision events. This deep learning-based classification method is more robust and adaptable than the traditional threshold judgment method.

[0124] Optionally, the motion state sensor includes a speed sensor and a gyroscope, the speed sensor is mounted at a first predetermined position of the forklift, and is used to detect the driving speed of the forklift to obtain driving speed data; the gyroscope is mounted at a second predetermined position of the forklift, and is used to detect the rotation state of the forklift to obtain acceleration data and angular velocity data.

[0125] Optionally, the first predetermined position is a position near a controller on the forklift 2 , and the second predetermined position is a horizontal position below a seat on the forklift 2 .

[0126] In the forklift 2 collision detection system, the first predetermined position is selected as the vicinity of the controller (also known as: Electronic Control Unit) on the forklift 2, and the second predetermined position is selected as the horizontal position below the seat on the forklift 2. This design brings many technical benefits, which are specifically described as follows:

[0127] 1. Accurate data collection and processing

[0128] Near the controller: Installing the speed sensor near the ECU can ensure the timeliness and accuracy of data transmission. In addition, there are usually better electrical connections and shielding measures near the ECU, which helps to reduce electromagnetic interference and improve the stability of data acquisition.

[0129] Horizontal position under the seat: This position is relatively stable and not easily affected by the violent vibration during the travel of the forklift 2. At the same time, the area under the seat is also part of the structure of the forklift 2, and its vibration characteristics can better reflect the overall motion state of the forklift 2. Installing the speed sensor here can capture more real and reliable data, providing strong support for collision detection.

[0130] 2. Optimize system layout and cost

[0131] System layout: Installing the gyroscope near the ECU and under the seat is conducive to optimizing the overall layout of the system. The two locations are relatively concentrated, which facilitates data exchange and communication between the gyroscope and the ECU, reducing wiring complexity and cost. At the same time, this layout also facilitates system maintenance and upgrades.

[0132] Cost control: By properly selecting the sensor installation location, the system cost can be reduced while ensuring the detection effect. For example, by using the existing structure and electrical layout of the forklift 2, the need for additional mounting brackets and fixtures can be reduced, thereby reducing material costs and installation costs.

[0133] 3. Improve the sensitivity and accuracy of collision detection

[0134] Sensitivity: The area near the ECU and under the seat are areas that are easily affected by the forklift 2 in the event of a collision. Installing sensors at these locations can more sensitively capture the vibration and impact signals generated by the collision, thereby improving the sensitivity of collision detection.

[0135] Accuracy: Since the sensors are installed at key locations on the forklift 2 structure, they can more accurately reflect the actual state changes of the forklift 2 during the collision process. This accuracy is not only reflected in the immediate response when the collision occurs, but also in the accurate judgment of the type, degree and direction of the collision.

[0136] Optionally, the gyroscope is a six-axis gyroscope.

[0137] Therefore, the six-axis gyroscope is selected as the key component for detecting the motion state of the forklift 2, which has the following technical benefits:

[0138] (1) The six-axis gyroscope can not only measure the angular velocity (rotational speed around three axes) of the forklift 2, but also simultaneously measure the linear acceleration (acceleration along three coordinate axes). This multi-dimensional motion detection capability enables the system to more fully understand the dynamic behavior of the forklift 2, including its translational and rotational motion.

[0139] (2) Compared with single-axis or three-axis gyroscopes, six-axis gyroscopes can reduce the errors that may be introduced when measuring a single dimension by measuring angular velocity and linear acceleration at the same time. This mutual verification and fusion of multi-dimensional data helps to improve the accuracy and reliability of the overall measurement data.

[0140] (3) In a complex and ever-changing working environment, forklift 2 may encounter various vibrations, shocks, and disturbances. The multi-dimensional measurement capability of the six-axis gyroscope enables it to better adapt to these complex situations and reduce the impact of a single interference factor on system performance by integrating data from multiple dimensions, thereby enhancing the robustness of the system.

[0141] The expressions "first", "second", "the first" or "the second" used in various embodiments of the present disclosure may modify various components regardless of order and / or importance, but these expressions do not limit the corresponding components. The above expressions are only configured for the purpose of distinguishing an element from other elements. For example, a first user device and a second user device represent different user devices, although both are user devices. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element without departing from the scope of the present disclosure.

[0142] When one element (e.g., a first element) is referred to as being "(operably or communicatively) coupled" or "(operably or communicatively) coupled to" or "connected to" another element (e.g., a second element), it is understood that the one element is directly connected to the other element or the one element is indirectly connected to the other element via yet another element (e.g., a third element). Conversely, it is understood that when an element (e.g., a first element) is referred to as being "directly connected" or "directly coupled" to another element (the second element), no element (e.g., a third element) is interposed between the two.

[0143] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

Claims

1. An intelligent detection system for forklift collision based on multi-source state data and neural network model, characterized in that: include: A motion state sensor is used to detect the motion state of the forklift and obtain motion state data, wherein the motion state data includes driving speed data, acceleration data and angular velocity data; A vibration state sensor, used for detecting the vibration state of the forklift to obtain vibration state data; The vehicle-mounted data acquisition terminal is used to collect the motion state data and the vibration state data to obtain motion state sampling data and vibration state sampling data, wherein the motion state sampling data includes driving speed sampling data, acceleration sampling data and angular velocity sampling data; A data analysis server is deployed with a time-frequency analysis model, a dynamic parameter analysis model, a data fusion model, an information entropy analysis model, and a collision detection model, wherein: The time-frequency analysis model is used to transform the vibration state sampling data from the time domain to the frequency domain to obtain a vibration characteristic value sequence of the vibration state sampling data in the frequency domain; The dynamic parameter analysis model is used to calculate the driving speed gradient sequence, the acceleration change rate sequence, and the angular velocity change rate sequence according to the driving speed sampling data, the acceleration sampling data, and the angular velocity sampling data respectively; The data fusion model is used to fuse the vibration characteristic value sequence, the driving speed gradient sequence, the acceleration change rate sequence, and the angular velocity change rate sequence to obtain a fused data sequence; The information entropy analysis model is used to perform information entropy analysis on the fused data sequence to evaluate the information gain of the fused data sequence and determine whether the information gain reaches a preset information gain threshold. If so, the collision detection model is triggered to determine whether the forklift has a collision based on the fused data sequence; otherwise, the fused data sequence is amplified so that the information gain of the amplified fused data sequence reaches the preset information gain threshold. Wherein, the collision detection model includes a convolutional neural network model, a recursive neural network model, a fusion module, and a classifier; The convolutional neural network model includes an input layer, a multi-layer convolution and a pooling layer; Converting the fused data sequence into an input vector of the multi-layer convolution based on the input layer; Performing a convolution operation on the input vector based on the multi-layer convolution to extract local spatial features in the vibration state data; Based on the pooling layer, the local spatial features are subjected to dimensionality reduction processing to obtain spatial features; The recurrent neural network model includes a serialization module and a gated recurrent unit; Based on the serialization module, vectorization processing is performed on the fused data sequence to obtain a time series feature vector; Based on the gated recurrent unit, feature extraction is performed on the time series feature vector to capture the time series features in the motion state sampling data; Based on the fusion module, the spatial features and the temporal features are fused to obtain fusion features; Based on the classifier, the fusion feature is mapped to a predefined category label to determine whether the forklift has a collision.

2. The intelligent detection system for forklift collision based on multi-source state data and neural network model according to claim 1, characterized in that: The vehicle-mounted data acquisition terminal is provided with a data interface, and the data interface is connected to the motion state sensor and the vibration state sensor to acquire the motion state data and the vibration state data.

3. The intelligent detection system for forklift collision based on multi-source state data and neural network model according to claim 2 is characterized in that: When the data interface is an analog quantity interface, the motion state data and the vibration state data are analog quantities; when the data interface is a digital quantity interface, the motion state data and the vibration state data are digital quantities.

4. The intelligent detection system for forklift collision based on multi-source state data and neural network model according to claim 3 is characterized in that: The data fusion model is used to perform the following steps to fuse the vibration characteristic value sequence, the driving speed gradient sequence, the acceleration change rate sequence, and the angular velocity change rate sequence to obtain a fused data sequence: Performing time stamp alignment on the vibration characteristic value sequence, the driving speed gradient sequence, the acceleration change rate sequence, and the angular velocity change rate sequence to obtain a synchronized vibration characteristic value sequence, driving speed gradient sequence, acceleration change rate sequence, and angular velocity change rate sequence; Recursively filtering the synchronized vibration characteristic value sequence, driving speed gradient sequence, acceleration change rate sequence, and angular speed change rate sequence to obtain calibrated vibration characteristic value sequence, driving speed gradient sequence, acceleration change rate sequence, and angular speed change rate sequence; The calibrated vibration characteristic value sequence, driving speed gradient sequence, acceleration change rate sequence, and angular velocity change rate sequence are fused to obtain a fused data sequence.

5. The intelligent detection system for forklift collision based on multi-source state data and neural network model according to claim 4, characterized in that: The synchronized vibration eigenvalue sequence, driving speed gradient sequence, acceleration rate sequence, and angular velocity rate sequence are respectively mapped to state estimates according to time steps, and the state estimates are recursively filtered based on the following formula to obtain updated state estimates, so as to obtain calibrated vibration eigenvalue sequences, driving speed gradient sequences, acceleration rate sequences, and angular velocity rate sequences, respectively: , used to estimate the state of the previous time step and the system control input To predict the state of the current time step ,in, A It is a state transfer matrix, which is used to describe the trend estimation of the vibration value, driving speed, acceleration, and angular velocity of the forklift over time when a collision occurs. B is the control input matrix, which is used to describe the influence weight of the control input on the vibration value, driving speed, acceleration and angular velocity of the forklift when a collision occurs; , used to calculate the covariance matrix of the predicted state , which measures the uncertainty of the predicted state, where is the state estimate covariance matrix of the previous time step, is the covariance matrix of the process noise; , the Kalman gain is calculated based on the state estimate covariance matrix of the previous time step , which determines the weights of the predicted state and the observed value when updating the state estimate, is the observation matrix, is the observation noise covariance matrix; , used to calculate the observed value based on the current time step To update the state estimate for the next time step , combined with the state of the current time step and observations , where the Kalman gain Controlled the weight of these two; , used to update the covariance matrix of the state estimate , which reflects the uncertainty of the updated state estimate.

6. The intelligent detection system for forklift collision based on multi-source state data and neural network model according to claim 5, characterized in that: The step of fusing the calibrated vibration characteristic value sequence, the driving speed gradient sequence, the acceleration change rate sequence, and the angular velocity change rate sequence to obtain a fused data sequence includes: Based on the prior probability distribution and conditional probability distribution of the motion state of the forklift, according to the calibrated vibration characteristic value sequence, driving speed gradient sequence, acceleration change rate sequence, and angular velocity change rate sequence, determine the vibration posterior probability distribution data of the forklift, the driving speed posterior probability distribution data, the acceleration posterior probability distribution data, and the angular velocity posterior probability distribution data of the forklift; The vibration posterior probability distribution data of the forklift, the driving speed posterior probability distribution data, the acceleration posterior probability distribution data, and the angular velocity posterior probability distribution data of the forklift are fused to obtain a fused data sequence.

7. The intelligent detection system for forklift collision based on multi-source state data and neural network model according to claim 6, characterized in that: The collision detection model determines whether the forklift has collided based on the fused data sequence, including: the collision detection model extracts features from the fused data sequence to obtain spatial features and temporal features, and fuses them to obtain fused features, so as to determine whether the forklift has collided by mapping the fused features to the constructed category labels.

8. The intelligent detection system for forklift collision based on multi-source state data and neural network model according to claim 7, characterized in that: The motion state sensor includes a speed sensor and a gyroscope, and the speed sensor is mounted at a first predetermined position of the forklift and is used to detect the driving speed of the forklift to obtain driving speed data; The gyroscope is mounted on a second predetermined position of the forklift and is used to detect the rotation state of the forklift to obtain acceleration data and angular velocity data.

9. The intelligent detection system for forklift collision based on multi-source state data and neural network model according to claim 8, characterized in that: The first predetermined position is a position near a controller on the forklift, and the second predetermined position is a horizontal position below a seat on the forklift.