Tower Crane Driver Behavior Evaluation System Based on Trajectory Reconstruction and Dynamic Time Warping

Abstract

The present invention provides a tower crane driver behavior evaluation system based on trajectory reconstruction and dynamic time warping, including an industrial control computer, a number of sensors, a display, a cloud server, and a wireless communication module. The main software part includes a local accuracy discrimination system based on trajectory reconstruction and an overall similarity discrimination model based on dynamic time warping. The behavior evaluation model is generated by training with historical data and updated and optimized with real-time data. It displays the three-dimensional spatial trajectory and real-time behavior evaluation results on the cab screen and during cloud access, can timely warn of abnormal operation behaviors, and quantitatively judge the operation behaviors, thereby improving the efficiency of driver training and assessment and realizing the objectivity and standardization of the operation evaluation mechanism.

Description

Technical Field

[0001] The invention belongs to the field of crane human reliability, and in particular relates to a tower crane driver behavior evaluation system based on trajectory reconstruction and dynamic time warping. Background Art

[0002] The safety and reliability of tower cranes have always been a hot topic and focus of research in this field. With the advancement of technology, the proportion of accidents caused directly by hardware reasons has been significantly reduced, and the dangerous driving behavior of tower crane drivers has become the primary link in the risk of tower crane accidents. The driving of tower cranes is a special operation behavior, which requires strict training before entering the production environment. The traditional training method for most tower crane drivers is that the instructor guides the trainees and teaches by word of mouth and by example. This training method has many human factors, the assessment and scoring are not objective, and it consumes a lot of manpower costs; the assessment and evaluation of drivers by production enterprises are mostly based on subjective scoring by experienced drivers or supervisors, and through accident result evaluation. It is often a post-evaluation method and cannot provide timely warnings for dangerous operating behaviors. Intelligent evaluation of driver operations and warning of dangerous behaviors are important technical issues that need to be solved in industry training and production. Summary of the invention

[0003] Purpose of the invention: The purpose of the present invention is to realize the automated evaluation of driver operation and early warning of dangerous behavior. By outputting the overall similarity evaluation index and the local accuracy evaluation index, the driver's operation situation is automatically evaluated and the score is predicted, and the driver's operation behavior is quantitatively judged, thereby improving the efficiency of driver training and assessment, and realizing the objectivity and standardization of the operation evaluation mechanism. Through trajectory reconstruction, the driver's behavior is decomposed, the driver's operation trajectory is simulated, and the virtual model established with the on-site environment is interacted with to determine whether the driver's operation trajectory crosses the boundary, and the local accuracy of the driver's operation is judged. The standard operation model is trained according to historical data, and the similarity between each driver and the standard operation model in the same operation process is compared. The overall evaluation of the driver's operation is given by the similarity of the time series, and the overall similarity of the driver's behavior is quantified.

[0004] The present invention specifically provides a tower crane driver behavior evaluation system based on trajectory reconstruction and dynamic time warping, including an industrial computer, a sensor, a display, a cloud server, and a behavior evaluation algorithm module;

[0005] The three-dimensional trajectory reconstruction is realized through the industrial computer, and it interacts with the simulation environment to identify the over-limit and over-height driving stages in the driver's operation, and the display shows the abnormal operation reminder in real time;

[0006] The behavior evaluation algorithm module calculates the Dynamic Time Warping (DTW) distance δ through the standard operation time series established based on the simulated three-dimensional model, outputs the overall similarity index σ, inputs the linear model generated by training on the historical model, predicts the operation score S, and realizes the automatic evaluation of the driver's behavior. π , and realizes the automatic evaluation of the driver's behavior by outputting the overall similarity index σ, inputting the linear model generated by training on the historical model, and predicting the operation score S.

[0007] The sensors include a height sensor, an amplitude sensor, and a slewing angle sensor;

[0008] The sensors and the display are connected to the industrial control computer through an industrial bus;

[0009] The industrial control computer accesses the gateway and is connected to the cloud server through the TCP / IP protocol to realize data storage and Internet access.

[0010] The sensors are used to collect data, which after preprocessing, are stored in the local MySQL server. Through the training and prediction of the behavior evaluation algorithm module, the three-dimensional trajectory and the marked operation segments are displayed on the display;

[0011] The industrial control computer includes a wireless communication module. The industrial control computer sends the data collected by the sensors in real time and the model calculation results to the cloud server through the wireless communication module to realize data storage and Internet access, distributes and presents the three-dimensional trajectory reconstruction map and the operation segment recognition results to the user through the cloud server, and implements controllable management and warning of the driver's state.

[0012] The behavior evaluation algorithm module obtains the real-time operation data of the driver collected by the sensors. The real-time operation data of the driver includes the lifting height, the luffing distance, and the slewing angle of the tower crane. Different drivers and different driving stages are divided according to the real-time operation data of the driver, specifically including:

[0013] Step a1, identify the change points with different window statistical attributes as segmentation points through the sliding window algorithm;

[0014] Step a2, perform K-means clustering;

[0015] Step a3, extract different sub-time series: divide the clustering results in step a1 into sub-time series of different drivers through a stationary threshold, and then for the sub-time series of a single driver, divide its clustering results into different driving stages according to the time sequence;

[0016] Step a4, repeat steps a1 to a3 until the sequences of different drivers and different driving stages are identified, and then reconstruct the three-dimensional trajectory.

[0017] Step a1 includes: The sliding window algorithm uses two windows that continuously step forward along the positive direction of time, and compares the statistical attributes and differences within each window to identify change points. For a given cost function c(), the derived difference metric function d() is:

[0018] d(y uv , y vw ) = c(y uw ) - c(y uv ) - c(y vw )

[0019] Define y t as the input signal, and t as the index coordinate of y t . Among them, let u < v < w be three different indices; y uv represents the sliding window from y u to y v , y uw represents the sliding window from y u to y w , y vw represents the sliding window from y v to y w .

[0020] The difference metric function is the cost gain divided between subsequences; if the sliding window and both belong to one segment, it means their statistical attributes are similar, and the difference between the first window and the second window is very small and not recorded as a change point. If the sliding window falls into two different segments, the difference is significantly higher, and the boundary between the windows is output as a change point.

[0021] Step a2 includes:

[0022] Step a2-1: Take the segment mean of the signals of the four sensors in each segment, namely the height sensor, amplitude sensor, rotation angle sensor, and load sensor, as features, use these features as the feature vectors of each sample, and randomly initialize the cluster centers;

[0023] Step a2-2: Assign each sample to its nearest cluster center, create new cluster centers by obtaining the average of all samples assigned to each previous cluster center, and calculate the difference between the old cluster centers and the new cluster centers;

[0024] Step a2-3: Repeat steps a2-1 to a2-2 until the difference value between the old cluster centers and the new cluster centers is less than the threshold (the threshold is generally 0.05), and the optimization objective is to minimize its sum of squares:

[0025]

[0026] Let the training sample be D, and x i be the i-th training sample; let the set of clustering center clusters be C, and μ j be the j-th sample clustering center.

[0027] The three-dimensional trajectory reconstruction implemented by the industrial control computer includes:

[0028] Calibrate and perform spatial transformation on the sensor data: The initial value of the lifting height is denoted as H1, the initial value of the luffing distance is denoted as R1, the initial value of the slewing angle is denoted as A1, and the initial value of the lifting weight is denoted as W1; the calibration starting values of the sensors are respectively denoted as the calibration height H0, the calibration luffing distance R0, the calibration slewing angle A0, and the calibration lifting weight W0; its spatial model coordinate system is (R, A, H) T , where R is the polar radius, A is the polar angle, and H is the height coordinate; the rectangular coordinate system is denoted as (X, Y, H) T , where X is the abscissa, Y is the ordinate, and H is the height coordinate; transform the spatial model (R, A, H) T to the rectangular coordinate system (X, Y, H) T , for each vector (R i , A i , H i , W i ) T , the transformed spatial vector is:

[0029]

[0030] where R i is the abscissa before spatial transformation, A i is the ordinate before spatial transformation, H i is the height coordinate before spatial transformation, H i , is the lifting weight value before spatial transformation; X′ i is the abscissa after spatial transformation, Y i ′ is the ordinate after spatial transformation, H′ i is the height coordinate after spatial transformation, W′ i is the lifting weight value after spatial transformation.

[0031] The behavior evaluation algorithm module is also used for the discrimination of the abnormal driving stage, specifically including the following steps:

[0032] By simulating the interaction between the three-dimensional trajectory and the environment, when the driver manipulates the hook to reach the predetermined position, if it exceeds the boundary of the simulated rod path, compare it with the boundary dynamic threshold based on historical data (the general threshold is taken as 0.3). If it exceeds the threshold, output the abnormal segment and calculate the behavior accuracy θ, otherwise update the historical data.

[0033] Among them:

[0034]

[0035] V1 represents the number of over-limit vectors within a segment, and V2 represents the total number of vectors within a segment.

[0036] The behavior evaluation algorithm module also performs the following steps:

[0037] Step b1: Based on historical data and standard operation examples, establish a standard operation model X. This model is obtained by having different skilled tower crane drivers perform a complete operation process in the corresponding scenario, and collecting operation data through multi-source sensors to establish a driving time series, with the average result as the initial model; and updating the model through better driving time data during actual training and operation, with the average result as the standard model.

[0038] Step b2: Calculate the dynamic time warping (DTW) distance between the driving time series Y of different drivers and the standard model as a similarity characterization of the same standard operation;

[0039] Step b3: Train a linear model based on historical operation data. For the linear model y = b0 + b1x1 + … + b k x k + u, where (z1, z2, …, x k ) are the features described by k given attributes, (b1, b2, …, b k ) are the weight coefficients of (x1, x2, …, x k ), and u is the bias of the linear model; its training process includes: initializing the model parameters b k and u. The training samples include the dynamic time warping (DTW) distance x calculated in step b2 and the historical evaluation score y. The specific training includes forward propagation calculation, error calculation, gradient calculation, and parameter update;

[0040] Step b4: Predict the score of the real-time collected data to achieve automatic evaluation and scoring of the operation: For the collected sensor data, calculate the dynamic time warping (DTW) distance, and after normalization, input it into the linear model established in step b3 to output the predicted operation score S.

[0041] Step b2 includes: For two transient sequences X = (x1, x2, …, x N ) and Y = (y1, y2, …, y M ) with lengths N and M respectively, x N is the Nth vector of sequence X, and y M is the Mth vector of sequence Y; in the feature space , for it means n ∈ [1:N] and m ∈ [1:M];

[0042] For two different features For the constructed cost matrix C ∈ R N×M , C(n,m) := c(x n , y m ), where R is the set of real numbers, C(n,m) is the element in the n-th row and m-th column of the N×M cost matrix C, and c(x n , y m ) is the cost function of x n and y m . The optimization objective is to find the optimal alignment of X and Y with the lowest cost, that is, to find the best matching path π = [π0, …, π K , where π K is the end point of the matching path π; the optimization objective of dynamic time warping (DTW) is expressed as:

[0043]

[0044] The size of the best matching path π is the DTW distance between the driver's operation sequence and the standard model, that is, a characterization of the similarity to the standard operation.

[0045] The system of the present invention obtains the current slewing angle, luffing trolley position, hook position information, lifting weight, and time stamp of the tower crane through a slewing angle sensor, a luffing trolley sensor, a height sensor, and a load sensor respectively. Resampling and standardizing the original data as preprocessing, then using the sliding window method to detect and divide the change points, and clustering the divided stages. The first step realizes the division of the operation time series and intermediate states of different drivers, and the second step realizes the subdivision of the operation time series of a single driver into different driving stages.

[0046] According to the on-site environment, a virtual model is established and simulated hoisting of the three-dimensional trajectory is carried out. When the driver manipulates the hook to reach the predetermined position, if it is lower than the ground height of the environmental virtual model, it is judged that the operating height is too low; if it is higher than the sum of the height of the environmental virtual model and the height of the lifted load, it is judged that the operating height is too high; if the hook exceeds the boundary conditions on both sides of the running track of the environmental virtual model, the overrun time points are accumulated and judged as an overrun of the operating position. For the overrun and ultra-high offset stages, they are output to the statistical results and marked as red segments in the three-dimensional trajectory diagram.

[0047] The three-dimensional trajectory model can compare the operations of different drivers, output the height accuracy situation and the position overrun offset situation, realize the automatic judgment of the driver's operation accuracy, can be used as an evaluation index for the local accuracy of the operation, and give early warning prompts for abnormal segments.

[0048] After the data collected by the multi-source sensors are processed by the industrial control computer, the abnormal segmentation models of different drivers and the overall similarity scores of operations can be obtained. Through the front-end module, the output is realized and displayed on the monitor, and the three-dimensional trajectory of the operation can be displayed in real time, and the operation offset segments can be marked in red, and the predicted automated driving operation score can be displayed.

[0049] Compared with the prior art, the present invention has the following beneficial effects: The automated operation evaluation and warning system improves the training efficiency and the reliability of driver behavior management, and enhances the intelligence and human-machine collaboration ability of the construction machinery operation system; The complexity of the construction industrial site construction environment, the instability of high-altitude Internet transmission, and the immediate need for operation evaluation and warning determine that the edge-cloud collaborative working mode has advantages. Necessary data processing and analysis are carried out at the terminal, and then the results are distributed and presented to users through the cloud server, which greatly reduces the dependence on cloud computing resources and local devices, has a fast response ability, and effectively reduces the risk brought by communication interruption. It also avoids the data risk of only local processing and storage, and ensures the traceability and reliability of driver operation data. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The following further detailed description of the present invention will be made in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become clearer.

[0051] Figure 1 It is a schematic diagram of the system structure.

[0052] Figure 2 It is a block diagram of the overall algorithm principle.

[0053] Figure 3 It is a block diagram of the abnormal trajectory discrimination process.

[0054] Figure 4 It is a schematic diagram of an abnormal trajectory discrimination example.

[0055] Figure 5 It is a block diagram of the overall similarity evaluation.

[0056] Figure 6 It is a segmented schematic diagram of different driving stages of the driver.

[0057] Figure 7 It is a schematic diagram of the instance field competition environment.

[0058] Figure 8 It is the optimal matching path of dynamic time warping.

[0059] Figure 9 It is a schematic diagram of the regression prediction of the linear model for the training set. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0060] Taking the data collected in the "4th National Prefabricated Building Vocational Skills Competition in 2021" as an example, the specific implementation manner of the present invention will be described by a specific example. The tower crane sensor data platform recorded approximately 500,000 tower crane operation monitoring data generated by 5 tower cranes during the competition on October 13, 2021 by 112 tower crane drivers, and recorded data such as the lifting height of the tower crane hook, trolley luffing, slewing angle of the jib, lifting weight, and generation time. The on-site competition environment diagram is as Figure 7 shown.

[0061] The specific algorithm implementation process is as Figure 2 , Figure 5 shown, and it is divided into a preprocessing stage, a data partitioning stage, a three-dimensional trajectory simulation interaction stage, and an overall similarity evaluation stage.

[0062] In the data preprocessing stage, the resampling averaging method is used to resample the original data at the same frequency. Then, the sliding window method is used to detect and partition the change points, and clustering is performed on the partitioned stages to achieve the partitioning stage of the original data: the first step is to achieve the partitioning of the operation time series and intermediate states of different drivers, and the second step is to subdivide the operation time series of a single driver into different driving stages. Here, an example is given to partition the change point set of the sensor data collected by Tower Crane No. 3 using the sliding window method, construct the mean values of the four sensor signals of each segment as features, and then perform principal component analysis (PCA) on these four features. Their variance contribution rates are 0.46757185, 0.26487326, 0.21187442, and 0.05568047 respectively. This linear dimensionality reduction process uses singular value decomposition of the data and projects it into a lower-dimensional space. The remaining tower cranes also perform the partitioning of the driver subsequences according to the above calculation method. After performing K-means clustering on the overall segments, different driver sub-time series are divided through a stationary threshold for the clustering results, and then the sub-time series of different drivers are extracted respectively. For the sub-time series of a single driver, its clustering results are divided into different driving stages according to the time sequence, and different stages are marked, such as Figure 6 shown, Figure 6 in which, Height represents the height coordinate of the time series curve, Angle represents the slewing angle value, Radius represents the slewing radius value, and Weight represents the weight of the lifting weight. For the preprocessing and partitioning of the original data, it is possible to perform subsequence partitioning on the multi-source sensor data of each tower crane, identify the driving stages of different drivers, and continue to partition the sequence of a single driver to distinguish different driving stages of the driver, providing a basis for the establishment of a three-dimensional model of the subsequent driver operation trajectory.

[0063] The three-dimensional trajectory simulation interaction stage is divided into three steps: the first is the reconstruction of the three-dimensional trajectory, the second is the interaction with the simulation environment, and the third is the discrimination of abnormal trajectories.

[0064] The initial value of the lifting height is denoted as H1 = 0.3, the initial value of the luffing distance is denoted as R1 = 12, the initial value of the slewing angle is denoted as A1 = 180, and the initial value of the suspended load is denoted as W1 = 0; the calibration starting values of the sensors are respectively denoted as the calibration height H0 = 0.3, the calibration luffing distance R0 = 12, the calibration slewing angle A0 = 137°, and the calibration suspended load W0 = 0.1. For the reconstruction of the three-dimensional spatial trajectory, the spatial model (R, A, H) T is transformed into the Cartesian coordinate system (X, Y, H) T , for each vector (R i , A i , H i , W i ) T in the original data, the transformed spatial vector is as follows:

[0065]

[0066] Taking the simulation environment data of the "Fourth National Prefabricated Building Vocational Skills Competition in 2021" as an example, the position of the collision block and the rod track model are established as the simulation environment. Its rod track model can be established as:

[0067]

[0068] The flow chart for discriminating abnormal trajectories by establishing a spatial simulation is as Figure 3 shown. The judgment steps for marking abnormal segments are as follows: By simulating the interaction between the three-dimensional trajectory and the environment, when the driver manipulates the hook to reach the predetermined position, first, if it exceeds the boundary of the simulated rod track, it is compared with the dynamic threshold of the boundary based on historical data; second, if the height exceeds the boundary of the simulated height, it is compared with the dynamic threshold of the height based on historical data; if the threshold is exceeded, the abnormal segment is output and the behavior accuracy is calculated, and if the threshold is not exceeded, the historical data is updated. The discrimination result and the example diagram of the three-dimensional trajectory reconstruction in this example are as Figure 4 shown.

[0069] In the overall similarity discrimination stage, first, the driving sensor data is collected according to the competition standard operation, and the standard operation model X is established based on this. The dynamic time warping distance is calculated between the operation time series Y of different drivers and it: For two transient sequences X = (x1, x2,..., x N ) and Y = (y1, y2,..., y M ) with lengths N and M respectively, x N is the last vector of sequence X, and y M is the last vector of sequence Y. In the feature space , for it means n ∈ [1:N] and m ∈ [1:M]. For two different features For the constructed cost matrix C ∈ R N×M , C(n, m): = c(x n , y m ), where C(n, m) is the (n, m) element of the N×M cost matrix C, and c(x n , y m ) is the cost function of x n and y m . The optimization objective is to find the alignment of X and Y with the optimal lowest cost, that is, to find the best matching path π = [π0, …, π K , where π K is the end point of the matching path π. The optimization objective of dynamic time warping DTW can be expressed as:

[0070]

[0071] The size of the best matching path π is the dynamic time warping DTW distance between the driver's operation sequence and the standard model, that is, the similarity characterization with the standard operation. In the example, here the driver with ID = 8 of Tower Crane No. 3 performs dynamic time warping calculation with the standard operation model, and its optimal matching path is as Figure 8 shown. This figure is the matching path diagram of multi-dimensional dynamic time warping, Figure 8 in which Height represents the height coordinate of the time series curve, Angle represents the slewing angle value, Radius represents the slewing radius value, and its dynamic time warping calculation result is 2.305.

[0072] Calculate the dynamic time warping distance between each driver's time series and the standard model, and use it as a regression feature, and use the manually scored score as the label of the linear model, then a linear model can be trained to predict the score of the real-time collected data and realize the automatic evaluation and scoring of the operation. The regression prediction schematic diagram of the linear model trained here is as Figure 9 shown. It is the least squares regression diagram, Figure 9 in which the abscissa MDDTW Distance is the dynamic time warping distance, the ordinate Competition Score is the competition score. In the legend, predict mean represents the regression prediction mean of the linear model, observation represents the sample observation value, and predict std represents the prediction variance. Its residual SSE is 55.997, the significance P(F) is 0.000, the mean absolute error MAE is 0.587, and the mean square error MSE is 0.615.

[0073] As Figure 1As shown in the figure, the present invention provides a tower crane driver behavior evaluation system based on trajectory reconstruction and dynamic time warping, which is divided into an edge hardware layer, a software layer and a web service layer. The sensor is used to collect data, which is preprocessed and stored in a local MySQL server. Through the training and prediction of the behavior evaluation algorithm module, the three-dimensional trajectory and the marked operation segments are displayed on the display. The industrial control computer includes a wireless communication module. Through the wireless communication module, the industrial control computer sends the data collected by the sensor in real time and the model calculation results to the cloud server to realize data storage and Internet access, and distributes and presents the three-dimensional trajectory reconstruction diagram and the operation segment recognition results to the user through the cloud server, and implements controllable management and warning of the driver's state.

[0074] In this specific implementation scheme, the real-time display of the operation trajectory and abnormal operation situations is realized, the automation and visualization of operation evaluation are realized, the competition scores can be analyzed and predicted through the similarity distance index, the intelligent evaluation of the training results is realized, and technical means are provided for the controllable management and warning of the training and the driver's state.

[0075] The present invention provides a tower crane driver behavior evaluation system based on trajectory reconstruction and dynamic time warping. There are many methods and ways to specifically implement this technical solution. The above is only the preferred implementation mode of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and retouches can be made, and these improvements and retouches should also be regarded as the protection scope of the present invention. Each component not clearly defined in this embodiment can be implemented by existing technologies.

Claims

1. A tower crane driver behavior evaluation system based on trajectory reconstruction and dynamic time warping, characterized in that, It includes an industrial control computer, sensors, a display, a cloud server, and a behavior evaluation algorithm module; Among them, the industrial control computer is used to realize three-dimensional trajectory reconstruction, interact with the simulation environment, identify the over-limit and ultra-high driving stages in the driver's operation, and display in real time and give abnormal operation reminders through the display; The behavior evaluation algorithm module calculates the dynamic time warping distance δ through the standard operation time series established based on the simulated three-dimensional model π , outputs the overall similarity index σ, inputs the linear model generated by training on the historical model, predicts the operation score S, and realizes the automatic evaluation of the driver's behavior; The behavior evaluation algorithm module obtains the real-time operation data of the driver collected by the sensors. The real-time operation data of the driver includes the lifting height, luffing distance, and slewing angle of the tower crane. Different drivers and different driving stages are divided according to the real-time operation data of the driver. Specifically, it includes: Step a1, identify the change points with different window statistical attributes as segmentation points through the sliding window algorithm; Step a2, perform K-means clustering; Step a3, extract different sub-time series: divide the clustering results in step a1 into sub-time series of different drivers through a stationary threshold, and then for the sub-time series of a single driver, divide the clustering results into different driving stages according to the time sequence; Step a4, repeat steps a1 to a3 until the sequences of different drivers and different driving stages are identified, and then perform three-dimensional trajectory reconstruction; The behavior evaluation algorithm module also performs the following steps: Step b1, establish a standard operation model X according to historical data and standard operation examples; Step b2, calculate the dynamic time warping distance between the driving time series Y of different drivers and the standard model as a similarity representation with the standard operation; Step b3: Train a linear model based on historical operation data. For the linear model y = b0 + b1x1 + … + b k x k + u, where (x1, x2, …, x k ) are the features described by k given attributes, (b1, b2, …, b k ) are the weight coefficients of (x1, x2, …, x k ), and u is the bias of the linear model; The training process includes: initializing the model parameters b k and u, and the training samples include the dynamic time warping distance and the historical evaluation score y calculated in step b2. Step b4, predict the scores of the real-time collected data to achieve automatic evaluation and scoring of operations: for the collected sensor data, calculate the dynamic time warping distance, and after normalization, input it into the linear model established in step b3 to output the predicted operation score S; Step b2 includes: for two transient sequences X = (x1, x2, …, x N ) and Y = (y1, y2, …, y M ) with lengths N and M respectively, x N is the Nth vector of sequence X, and y M is the Mth vector of sequence Y; in the feature space for it is expressed that n ∈ [1:N] and m ∈ [1:M]; For two different features For the constructed cost matrix C ∈ R N×M , C(n,m) := c(x n , y m ), where R is the set of real numbers, C(n,m) is the element in the n-th row and m-th column of the N×M cost matrix C, and c(x n , y m ) is the cost function of x n and y m ; The optimization objective is to find the optimal alignment of X and Y with the lowest cost, that is, to find the best matching path π = [π0, …, π K , where π K is the end point of the matching path π; the optimization objective of dynamic time warping is expressed as: The size of the best matching path π is the dynamic time warping distance between the driver's operation sequence and the standard model, that is, the similarity representation with the standard operation.

2. The system according to claim 1, wherein The sensors include a height sensor, a range sensor, and a slewing angle sensor; The sensors and the display are connected to the industrial control computer through an industrial bus; The industrial control computer accesses the gateway and is connected to the cloud server through the TCP / IP protocol to achieve data storage and Internet access.

3. The system according to claim 2, characterized in that, The sensors are used to collect data, which is preprocessed and stored in a local server. Through the training and prediction of the behavior evaluation algorithm module, the three-dimensional trajectory and marked operation segments are displayed on the display; The industrial control computer includes a wireless communication module. The industrial control computer sends the data collected by the sensors in real time and the model calculation results to the cloud server through the wireless communication module to achieve data storage and Internet access, distributes and presents the three-dimensional trajectory reconstruction map and the operation segment recognition results to users through the cloud server, and implements controllable management and warning of the driver's state.

4. The system according to claim 3, wherein Step a1 includes: The sliding window algorithm uses two windows that continuously step forward along the positive direction of time, and compares the statistical attributes and differences within each window to identify the change points. For a given cost function c(), the derived difference metric function d() is: d(y uv , y vw ) = c(y uw ) - c(y uv ) - c(y vw ) Define y t as the input signal, and t as the index coordinate of y t , where u < v < w are three different indices; y uv represents a sliding window from y u to y v , y uw represents a sliding window from y u to y w , and y vw represents a sliding window from y v to y w .

5. The system according to claim 4, characterized in that Step a2 includes: Step a2-1: Take the segment mean of the signals of the four sensors for each segment, namely the height sensor, amplitude sensor, slewing angle sensor, and load sensor, as features, use these features as the feature vectors of each sample, and randomly initialize the cluster centers; Step a2-2: Assign each sample to its nearest cluster center, create new cluster centers by obtaining the average of all samples assigned to each previous cluster center, and calculate the difference between the old cluster centers and the new cluster centers; Step a2-3: Repeat Step a2-1 to Step a2-2 until the difference value between the old cluster centers and the new cluster centers is less than the threshold, and the optimization objective is to minimize its sum of squares: Let the training sample be D, and x i be the i-th training sample; let the set of cluster centers be C, and μ j be the j-th sample cluster center.

6. The system according to claim 5, characterized in that The three-dimensional trajectory reconstruction implemented by the industrial control computer includes: Calibrate and perform spatial transformation on the sensor data: Denote the initial value of the lifting height as H1, the initial value of the luffing distance as R1, the initial value of the slewing angle as A1, and the initial value of the load weight as W1; Denote the calibration starting values of the sensors as the calibration height H0, the calibration luffing distance R0, the calibration slewing angle A0, and the calibration load weight W0; Its spatial model coordinate system is (R, A, H) T , where R is the polar radius, A is the polar angle, and H is the height coordinate; Denote the rectangular coordinate system as (X, Y, H) T , where X is the abscissa, Y is the ordinate, and H is the height coordinate; Transform the spatial model (R, A, H) T to the rectangular coordinate system (X, Y, H) T , for each vector (R i , A i , H i , W i ) T , the transformed spatial vector is: where R i is the abscissa before spatial transformation, A i is the ordinate before spatial transformation, H i is the height coordinate before spatial transformation, W i is the suspended load value before spatial transformation; X i ′ is the abscissa after spatial transformation, Y i ′ is the ordinate after spatial transformation, H i ′ is the height coordinate after spatial transformation, W i ′ is the suspended load value after spatial transformation.

7. The system according to claim 6, characterized in that The behavior evaluation algorithm module is also used for the discrimination of abnormal driving phases, specifically including the following steps: By simulating the interaction between the three-dimensional trajectory and the environment, when the driver manipulates the hook to reach the predetermined position, if it exceeds the boundary of the simulated rod path, compare it with the dynamic threshold of the boundary based on historical data. If it exceeds the threshold, output the abnormal segment and calculate the behavior accuracy θ, otherwise update the historical data; Where: V1 represents the number of over-limit vectors within the segment, and V2 represents the total number of vectors within the segment.

Images