Safety monitoring method and system for crane operating handle

Abstract

The invention provides a crane operating handle safety monitoring method and system. The method comprises the following steps: creating a safety monitoring object; setting monitoring nodes; acquiring handle operation habit data; obtaining mental state data of the driver; analyzing the fatigue state of the driver; setting a dynamic threshold value of handle operation; analyzing whether the handle operation habit data exceeds a threshold value or not by utilizing an HMM; and analyzing the judgment result in combination with the mental state data of the driver, and judging whether an abnormal condition exists or not. By establishing a comprehensive handle safety monitoring system, intelligent monitoring of handle operation is achieved, the problem of false alarm in handle operation safety monitoring is effectively solved by combining the mental state of a driver while the grip strength, the moving speed and the displacement of the handle are considered, the intelligent level and safety of handle safety monitoring are remarkably improved, and the safety of the handle operation is improved. And a more stable and reliable operation environment is provided for the operation of the crane handle.

Description

Technical Field

[0001] The present invention relates to the technical field of safety monitoring of crane operating handles, and in particular to a method and system for safety monitoring of crane operating handles. Background Art

[0002] In the modern industrial field, cranes are key equipment for heavy material handling, and their operational safety is directly related to production efficiency and personnel safety. With the advancement of science and technology, the safety monitoring of crane operations has received increasing attention, especially the real-time monitoring of handle operations has become an important means to improve safety. However, the existing technology still has significant defects in this field, which are mainly reflected in the following aspects: The monitoring data in the traditional crane handle safety monitoring system is single. For example, it often only focuses on the displacement of the handle as the basic operating parameter. This single data monitoring method is difficult to fully reflect the actual operating status of the operator, and it is also unable to capture the subtle anomalies that may exist in the handle operation. As a result, the accuracy of identifying abnormal handle operation is low, and it is difficult to adapt to the complex and changeable actual operating environment, resulting in frequent false alarms and missed alarms, which reduces the reliability of the system. And the influence of the driver's mental state is ignored: the safety of crane operation depends not only on the handle operation itself, but also on the driver's mental state. The driver's fatigue state can also cause abnormal handle operation, which in turn causes safety accidents. However, most of the existing technologies ignore this key factor and do not include the driver's mental state in the monitoring range, so that it is impossible to effectively warn of potential safety risks. In view of the above defects, there is an urgent need for a more comprehensive, accurate and intelligent crane operating handle safety monitoring method. Summary of the invention

[0003] Purpose of the invention: The present invention provides a crane operating handle safety monitoring method and system, the purpose of which is to solve the problems existing in the prior art.

[0004] Technical solution: The present invention proposes a crane operating handle safety monitoring method, the steps comprising: S1. Create safety monitoring objects, including handle operation habits and driver mental state; S2. Set up monitoring nodes: Set up multiple monitoring nodes for key locations of security monitoring objects; S3. Obtaining handle operation habit data: obtaining handle operation habit data in real time according to the monitoring node; the handle operation habit data includes grip force of the operating handle, handle movement speed, and handle displacement measured by the sensor; S4. Obtaining driver mental state data: obtaining driver mental state data in real time according to the monitoring node; the driver mental state data includes pupil state, blinking frequency and heart rate; S5. Analyze the driver's fatigue state: Analyze the driver's fatigue state based on the driver's mental state data obtained; the fatigue state is divided into three types: awake, mild fatigue and severe fatigue; S6. Setting dynamic thresholds for handle operation: setting dynamic thresholds for handle operation according to the historical data of handle operation habits; the handle dynamic thresholds include grip strength threshold, movement speed threshold, and displacement threshold; S7. Use HMM to analyze whether the handle operation habit data exceeds a threshold and obtain a judgment result.

[0005] S8. Analyze the determination result of S7 in combination with the driver's mental state data to determine whether there is an abnormal situation.

[0006] Furthermore, in S2, the main steps of setting the dynamic threshold are: S22. Grip strength threshold setting; S23. Speed ​​threshold setting; S24. Displacement threshold setting; Furthermore, in S2, a calibration period for the dynamic threshold is also set, and the main steps are: S25. Obtain the driver's operating habits data on the handle in the past M days; S26. Calculate the mean of historical data; S27. Calculate the standard deviation of historical data; S28. Update the dynamic threshold using the updated mean and standard deviation of grip force, movement speed, and displacement.

[0007] Further, the S5 comprises the following steps: S51. Construct a multimodal data set based on the obtained driver's pupil state, blinking frequency, head posture, and heart rate data; S52. Preprocess the multimodal data set to obtain a standardized input feature vector; S53. Extracting multimodal features of the driver's pupil state, eye movement features, head posture, and heart rate through DNN; S54. Input the obtained multimodal features into LSTM, and analyze the impact of the driver's historical state on the future fatigue state through time series modeling; S55. Based on the output result of the LSTM network, predict the driver's fatigue state in the future period of time.

[0008] Furthermore, the prediction of the fatigue state includes classifying the LSTM output using a softmax activation function to classify the driver's fatigue state into awake, mild fatigue, and severe fatigue.

[0009] Further, the S7 comprises the following steps: S71.HMM initialization; S72. Observational sequence analysis; S73. Calculate the probability of a given observation sequence using the forward algorithm; S74. Calculate the probability of a given observation sequence using the backward algorithm; S75. Use the Viterbi algorithm to determine the most likely hidden state sequence, given the current observation sequence; S76. Input the observation sequence into HMM, perform threshold determination, and identify abnormal operations.

[0010] Furthermore, the identifying abnormal operation includes: Non-subjective accidental touch: If the grip force does not reach the minimum threshold in the analysis of the handle operation habit data, it is determined to be a non-subjective accidental touch; Excessive stress operation: If the analyzed handle operation habit data exceeds the set upper limit threshold, it is judged as excessive stress operation.

[0011] Further, step S8 includes: S8.1: If the analyzed handle operation habit data does not reach the minimum threshold, it is determined to be a non-subjective conscious accidental touch, and the handle operation will not be responded to; S8.2: If the analyzed handle operation habit data exceeds the threshold, further determine the current driver fatigue state.

[0012] Further, step S8.2 includes: S8.2.1: If the driver is currently awake, remind him to pay attention to operating safety; S8.2.1: If the current driver is in a state of mild fatigue, the operation instruction will not be triggered, and the driver will be prompted to operate the handle again in the same steps to confirm that the current operation is normal; S8.2.1: If the current driver is in a state of severe fatigue, do not trigger the operation command, return the operating handle to zero position, and sound an alarm to remind the driver to change.

[0013] The present invention also provides a crane handle safety monitoring system, the system comprising: Data collection module: obtain handle operation habit data and driver mental state data; Data processing module: calculate, store, analyze and handle exceptions of collected data; Early warning module: used to issue an early warning when the handle is operated abnormally; The data visualization module is used to display the current handle operation grip, movement speed, displacement status, and driver fatigue status in real time. Beneficial Effects

[0014] 1. Compared with the prior art, and different from the traditional monitoring method that relies only on a single data source, the present invention establishes a comprehensive crane operating handle safety monitoring system, collects multimodal data of handle operation habits and the driver's mental state in real time for comprehensive analysis, and achieves a comprehensive and accurate evaluation of handle operation. This method significantly improves the accuracy of judging whether the handle operation is abnormal, can detect potential safety hazards earlier, intervene and correct them in time, effectively avoid safety accidents caused by driver fatigue or misoperation, and significantly improves the safety and reliability of crane operation.

[0015] 2. Compared with the prior art, the present invention introduces a dynamic threshold setting mechanism, which sets the threshold calibration according to the driver's historical operating habits to dynamically adjust the minimum and maximum thresholds of grip force, speed and displacement. This mechanism allows the monitoring system to take into account the influence of time factors to more accurately judge and identify abnormal operating behaviors, reduce false alarms and missed alarms, and improve the accuracy and effectiveness of monitoring.

[0016] 3. Compared with the prior art, the anomaly detection method based on the hidden Markov model (HMM) of the present invention can achieve in-depth mining of the implicit state sequence in the operation behavior by introducing HMM to model and analyze the handle operation data, so as to effectively identify the abnormal situations of non-subjective false touch and excessive stress operation. This anomaly detection method based on the probability statistical model has high robustness and anti-interference ability, and can maintain stable monitoring performance in complex and changeable working environments.

[0017] 4. Compared with the prior art, the present invention can distinguish the difference between non-subjective mis-touch, over-stress operation and normal operation with unprecedented accuracy by simultaneously monitoring the grip force, movement speed and displacement of the handle and fusing and analyzing these parameters. Not only the physical strength of the operation (such as grip force) and speed (such as movement speed) are taken into account, but also the spatial dynamics of the operation (such as displacement trajectory), thereby constructing a multi-dimensional operation recognition model. This model can capture and analyze subtle changes in operation in real time, and can more accurately identify abnormal operations including non-subjective mis-touch or over-stress operation, significantly improving the safety of crane operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flow chart of the safety monitoring method of the crane operating handle; Figure 2 It is a structural diagram of the crane operating handle safety monitoring system. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] Reference Figure 1 The present invention provides a crane operating handle safety monitoring method, and the specific steps and implementation methods are as follows: The present invention provides a crane operating handle safety monitoring method, the steps comprising: Create a safety monitoring object, including handle operation habits and driver's mental state, and set multiple monitoring nodes at key positions of the safety monitoring object. Obtain handle operation habit data in real time according to the monitoring node; the handle operation habit data includes the grip force of the operating handle, the handle movement speed, and the handle displacement measured by the sensor; obtain the driver's mental state data in real time according to the monitoring node; the driver's mental state data includes pupil state, blinking frequency, and heart rate. Analyze the driver's fatigue state based on the acquired driver's mental state data, specifically including constructing a multimodal data set based on the acquired driver's pupil state, blinking frequency, head posture, and heart rate data; preprocess the multimodal data set to obtain a standardized input feature vector; extract the driver's pupil state, eye movement features, head posture, and heart rate multimodal features through DNN; input the obtained multimodal features into LSTM, and analyze the impact of the driver's historical state on the future fatigue state through time series modeling; based on the output result of the LSTM network, predict the driver's fatigue state in the future period of time, and the prediction of the fatigue state includes using the softmax activation function to classify the LSTM output, and divide the driver's fatigue state into awake, mild fatigue, and severe fatigue.

[0021] Set the dynamic threshold of the controller according to the historical data of the controller's operation habits; Grip strength threshold setting: ; ; in, is the upper limit of grip strength, is the lower limit of grip strength, is the mean grip strength, is the standard deviation of grip strength, k is the safety factor; Speed ​​threshold setting: ; ; in, The upper limit of the movement speed. is the lower limit of the moving speed, is the mean moving speed, is the standard deviation of the moving speed, k is the safety factor; Displacement threshold setting: ; ; in, is the upper limit of displacement, is the lower limit of displacement, is the mean displacement, is the displacement standard deviation, k is the safety factor; To set the calibration period for the dynamic threshold, the main steps are: Obtain the driver's operating habit data on the handle in the past M days; Calculate the mean of historical data: ; in, is the mean value of m in the past M days, m= , ,in, For grip strength, For speed, For displacement Calculate the standard deviation of historical data : ;

[0022] in, is the standard deviation of m over the past M days, m= , ,in, For grip strength, For speed, For displacement.

[0023] The dynamic threshold is updated using the updated mean and standard deviation of grip force, movement speed, and displacement.

[0024] Finally, HMM is used to analyze whether the handle operation habit data exceeds the threshold, including the following steps: Initialization of HMM; including definition of HMM model parameters: state set (representing different operation modes, including normal operation, non-subjective false touch, and over-stress operation), observation sequence set (including observation data of handle grip force, speed, and displacement), initial state probability distribution, state transition probability matrix, and observation probability matrix. Pre-train the model parameters based on the acquired data; Observation sequence analysis: Real-time acquisition of handle operation habit data, including grip strength, speed, and displacement observations, to form an observation sequence. Preprocess the observation sequence, such as denoising and normalization, to improve the accuracy of subsequent analysis; Use the forward algorithm to calculate the probability of a given observation sequence; using the forward algorithm, starting from the initial state, based on the state transition probability and the observation probability, gradually calculate the probability and of the observation sequence under all possible state sequences, evaluate the matching degree of the model for the given observation sequence, and provide a basis for subsequent state inference; Use the backward algorithm to calculate the probability of a given observation sequence; starting from the last observation in the observation sequence, reversely calculate the probability of each state under a given subsequent observation sequence, and use it in conjunction with the forward algorithm to verify the consistency of the model or perform model optimization, providing another perspective to examine the model's ability to explain the observation sequence; Use the Viterbi algorithm to determine the most likely implicit state sequence, given the current observation sequence; Apply the Viterbi algorithm, based on the results of the forward calculation, to find the most likely state sequence through dynamic programming, which can best explain the given observation sequence and provide a direct basis for anomaly detection; The observation sequence is input into HMM, threshold judgment is performed, and abnormal operations are identified; the inferred most likely state sequence is compared with the preset threshold to obtain the judgment result.

[0025] The judgment result of HMM is combined with the driver's mental state data for analysis to determine whether there is an abnormal situation. The identification of abnormal operations includes: non-subjective false touch: if the grip force does not reach the minimum threshold in the analysis of the handle operation habit data, it is determined to be a non-subjective false touch; excessive stress operation: if the analysis of the handle operation habit data exceeds the set upper limit threshold, it is determined to be an excessive stress operation.

[0026] If the analyzed handle operation habit data does not reach the minimum threshold, it is judged as a false touch that is not subjective and conscious, and the operation of the handle will not be responded to; if the analyzed handle operation habit data exceeds the threshold, the current driver's fatigue state is further judged: if the current driver is awake, he is reminded to pay attention to operation safety; if the current driver is in a state of mild fatigue, the operation instruction is not triggered, and the driver is prompted to operate the handle again with the same steps to confirm that his current operation is normal; if the current driver is in a state of severe fatigue, the operation instruction is not triggered, the operating handle is returned to zero, and an alarm is issued to remind the driver to change the driver.

[0027] Reference Figure 2The present invention also proposes a crane handle safety monitoring system, which includes a data acquisition module, a data processing module, an early warning module and a data visualization module. Data acquisition module: This module can collect the operation habit data of the crane handle (such as grip strength, moving speed, displacement trajectory, etc.) and the driver's mental state data (such as heart rate, blinking frequency, facial expression, etc.) in real time and accurately, ensuring the comprehensiveness and timeliness of the data, and providing a data basis for subsequent analysis and early warning. Data processing module: Perform in-depth calculation and analysis on the collected data, identify abnormal operation modes and the driver's potential fatigue state, improve the system's recognition ability and processing efficiency for complex situations, and automatically mark and record the identified abnormal operations or driver fatigue states, providing a basis for subsequent safety training and improvement. Early warning module: Used to issue an early warning when the handle operation is abnormal. The data visualization module is used to display the current handle operation grip strength, moving speed, displacement state, and driver fatigue state in real time, which improves the intuitiveness and convenience of monitoring, and also provides managers with a wealth of data analysis tools to help them better understand the operation behavior mode and formulate more scientific and reasonable safety management and training strategies.

[0028] Based on the above embodiment, the working principle is: First, the method collects data in real time at the crane operating handle and the key parts of the driver through precisely arranged monitoring nodes. This process aims to fully capture the operating habits of the handle, including the size of the grip, the speed of the handle movement and the distance of displacement, while monitoring the driver's pupil state, blinking frequency and heart rate and other physiological indicators to accurately reflect the driver's mental state. The purpose of this step is to provide detailed and accurate basic data for subsequent data analysis to ensure the accuracy and reliability of monitoring.

[0029] Next, the collected multimodal data sets are deeply analyzed and processed using advanced deep learning algorithms, such as deep neural networks (DNN) and long short-term memory networks (LSTM). DNN is responsible for extracting high-level feature representations from multi-source data such as pupil status, eye movement characteristics, head posture, and heart rate, while LSTM uses these features to perform time series modeling and predict the driver's future fatigue state. The purpose of this step is to explore the deep information hidden behind the data, improve the accuracy and real-time nature of fatigue state prediction, and thus promptly discover potential safety hazards.

[0030] After analyzing the driver's mental state, the method dynamically sets thresholds for parameters such as grip force, speed, and displacement based on historical data of handle operation habits. These thresholds are not static, but are dynamically adjusted based on individual differences and operating habits of the driver to ensure the pertinence and effectiveness of monitoring. The purpose of setting dynamic thresholds is to make the monitoring system more intelligent and flexible, and to more accurately identify abnormal operating behaviors.

[0031] Subsequently, the Hidden Markov Model (HMM) was used to conduct an in-depth analysis of the handle operation habit data. HMM uses observation sequence analysis and the Viterbi algorithm to identify the implicit state sequence in the handle operation and determine whether there are abnormal situations such as non-subjective false touches and excessive stress operations. The purpose of this step is to use probabilistic statistical models to model and identify complex operating behaviors, thereby improving the accuracy and robustness of anomaly detection.

[0032] Finally, the handle operation habit data and the driver's mental state data are comprehensively analyzed and evaluated. If abnormal operation behavior is detected and the driver is in a fatigued state, the system will automatically take corresponding safety measures, such as reminding the driver to pay attention to operation safety, preventing the execution of dangerous operation instructions, or issuing an alarm and suggesting a driver replacement. The purpose of this step is to transform the monitoring results into practical actions, timely intervene and correct potential safety risks, and ensure the safety and reliability of crane operation. This method achieves comprehensive monitoring and effective protection of crane operation safety through the close cooperation and collaboration of multiple steps such as real-time data acquisition, deep learning analysis, dynamic threshold setting, HMM anomaly detection, and comprehensive safety response.

[0033] The crane handle safety monitoring system realizes comprehensive monitoring and abnormal warning of handle operation through the collaborative work of data acquisition module, data processing module, early warning module and data visualization module, which significantly improves the safety, efficiency and intelligence level of crane operation.

[0034] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present invention (including the claims) is limited to these examples. Under the concept of the present invention, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the present invention as described above, which are not provided in detail for the sake of simplicity.

[0035] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A crane operating handle safety monitoring method, characterized in that: The following steps are involved: S1. Create safety monitoring objects, including handle operation habits and driver mental state; S2. Set up monitoring nodes: Set up multiple monitoring nodes for key locations of security monitoring objects; S3. Obtaining handle operation habit data: obtaining handle operation habit data in real time according to the monitoring node; the handle operation habit data includes grip force of the operating handle, handle movement speed, and handle displacement measured by the sensor; S4. Obtaining driver mental state data: obtaining driver mental state data in real time according to the monitoring node; the driver mental state data includes pupil state, blinking frequency and heart rate; S5. Analyze the driver's fatigue state: Analyze the driver's fatigue state based on the driver's mental state data obtained; the fatigue state is divided into three types: awake, mild fatigue and severe fatigue; S6. Setting dynamic thresholds for handle operation: setting dynamic thresholds for handle operation according to the historical data of handle operation habits; the handle dynamic thresholds include grip strength threshold, movement speed threshold, and displacement threshold; S7. Use HMM to analyze whether the handle operation habit data exceeds the threshold and obtain a determination result; S8. Analyze the determination result of S7 in combination with the driver's mental state data to determine whether there is an abnormal situation.

2. A crane operating handle safety monitoring method according to claim 1, characterized in that: In S2, the main steps of setting the dynamic threshold are: S22. Grip strength threshold setting; S23. Speed ​​threshold setting; S24. Displacement threshold setting.

3. A crane operating handle safety monitoring method according to claim 2, characterized in that: In S2, a calibration period for the dynamic threshold is also set, and the main steps are: S25. Obtain the driver's operating habits data on the handle in the past M days; S26. Calculate the mean of historical data; S27. Calculate the standard deviation of historical data; S28. Update the dynamic threshold using the updated mean and standard deviation of grip force, movement speed, and displacement.

4. A crane operating handle safety monitoring method according to claim 1, characterized in that: The S5 comprises the following steps: S51. Construct a multimodal data set based on the obtained driver's pupil state, blinking frequency, head posture, and heart rate data; S52. Preprocess the multimodal data set to obtain a standardized input feature vector; S53. Extracting multimodal features of the driver's pupil state, eye movement features, head posture, and heart rate through DNN; S54. Input the obtained multimodal features into LSTM, and analyze the impact of the driver's historical state on the future fatigue state through time series modeling; S55. Based on the output result of the LSTM network, predict the driver's fatigue state in the future period of time.

5. A crane operating handle safety monitoring method according to claim 4, characterized in that: The prediction of the fatigue state includes classifying the LSTM output using a softmax activation function to classify the driver's fatigue state into awake, mild fatigue, and severe fatigue.

6. A crane operating handle safety monitoring method according to claim 1, characterized in that: The S7 comprises the following steps: S71.HMM initialization; S72. Observational sequence analysis; S73. Calculate the probability of a given observation sequence using the forward algorithm; S74. Calculate the probability of a given observation sequence using the backward algorithm; S75. Use the Viterbi algorithm to determine the most likely hidden state sequence, given the current observation sequence; S76. Input the observation sequence into HMM, perform threshold determination, and identify abnormal operations.

7. A crane operating handle safety monitoring method according to claim 6, characterized in that: The abnormal identification operation includes: Non-subjective accidental touch: If the grip force does not reach the minimum threshold in the analysis of the handle operation habit data, it is determined to be a non-subjective accidental touch; Excessive stress operation: If the analyzed handle operation habit data exceeds the set upper limit threshold, it is judged as excessive stress operation.

8. A crane operating handle safety monitoring method according to claim 1, characterized in that: Step S8 includes: S8.1: If the analyzed handle operation habit data does not reach the minimum threshold, it is determined to be a non-subjective conscious accidental touch, and the handle operation will not be responded to; S8.2: If the analyzed handle operation habit data exceeds the threshold, further determine the driver's fatigue status.

9. A crane operating handle safety monitoring method according to claim 8, characterized in that: Step S8.2 includes: S8.2.1: If the driver is currently awake, remind him to pay attention to operating safety; S8.2.1: If the current driver is in a state of mild fatigue, the operation instruction will not be triggered, and the driver will be prompted to operate the handle again in the same steps to confirm that the current operation is normal; S8.2.1: If the current driver is in a state of severe fatigue, do not trigger the operation command, return the operating handle to zero position, and sound an alarm to remind the driver to change.

10. A crane handle safety monitoring system, characterized in that: The system comprises: Data collection module: obtain handle operation habit data and driver mental state data; Data processing module: calculate, store, analyze and handle exceptions of collected data; Early warning module: used to issue an early warning when the handle is operated abnormally; The data visualization module is used to display the current handle operation grip, movement speed, displacement status, and driver fatigue status in real time.

Images