Object motion state monitoring method, medium and equipment

Abstract

The invention relates to the technical field of motion monitoring, and discloses an object motion state monitoring method, which comprises the following steps of: 1, acquiring and preprocessing motion state data of an object through an IMU (Inertial Measurement Unit) sensor; 2, training a deep learning model by using the preprocessed motion state data; 3, deploying the trained deep learning model to process the data collected in real time, and obtaining an IMU motion state monitoring result of the object; and step 4, outputting final motion state information of the object according to an IMU motion state monitoring result in combination with Beidou displacement monitoring information. According to the method, the motion state information is judged in multiple dimensions by using the IMU sensor, and the IMU data is optimized through deep learning, so that the IMU data is more suitable for monitoring the motion state, and the accuracy is improved. By assisting Beidou monitoring, a Beidou displacement monitor is helped to identify abnormal gross errors, equipment state information can be independently judged, and the monitoring reliability of Beidou displacement monitoring equipment is improved.

Description

Technical Field

[0001] The present invention relates to the field of motion monitoring technology, and in particular to a method, medium and device for monitoring the motion state of an object. Background Art

[0002] The BeiDou satellite navigation system plays a key role in monitoring geological disaster deformation. The system deploys high-precision GNSS receiving equipment to capture the navigation signals emitted by BeiDou satellites, and combines differential positioning technology with precise single-point positioning technology to achieve millimeter-level surface displacement monitoring accuracy. The system continuously collects three-dimensional coordinate data of monitoring points and establishes a spatiotemporal evolution model of surface deformation, thereby achieving early identification and early warning of potential geological disasters. This monitoring solution is characterized by all-weather, all-day, and high-precision, and can conduct real-time dynamic monitoring of geological disaster-prone areas such as landslides, ground subsidence areas, and fault zones.

[0003] However, in the existing high-precision Beidou displacement real-time monitoring process, due to the obstruction of buildings and tall trees, the satellite signal will have a certain quality change. At this time, the monitoring data may have large errors, which is easy to produce large error results, thereby causing certain judgment interference on the actual occurrence of disasters; at the same time, the Beidou equipment itself also has certain errors and is not sensitive to equipment status such as equipment jitter. If combined with conventional sensor monitoring methods, such as simply judging whether the device is moving by the threshold of the accelerometer and gyroscope sensor for auxiliary judgment, there will also be large gross errors, and it is impossible to accurately judge the actual movement status of the device.

[0004] Therefore, how to provide a method to improve the accuracy of IMU sensor motion state recognition to assist Beidou displacement monitoring equipment in object motion state recognition and monitoring is a technical problem that urgently needs to be solved. Summary of the invention

[0005] The object of the present invention is to provide a method for monitoring the motion state of an object, and its specific technical solution is as follows:

[0006] A method for monitoring the motion state of an object comprises the following steps:

[0007] Step 1: Obtain and pre-process the motion state data of the object through the IMU sensor;

[0008] Step 2: Use the preprocessed motion state data to train the deep learning model;

[0009] Step 3: Deploy the trained deep learning model to process the data collected by the IMU sensor in real time to obtain the IMU motion state monitoring results of the object;

[0010] Step 4: Based on the IMU motion state monitoring results and combined with the Beidou displacement monitoring information, output the object's final motion state information.

[0011] Preferably, the method further includes step five: inputting the final object motion state information into the trained deep learning model for feedback and triggering the deep learning model update.

[0012] Preferably, the step 1 comprises:

[0013] Collect motion state data of the object to be tested under different states;

[0014] Label and classify motion status data;

[0015] Perform data cleaning on the collected motion state data to remove noise and incomplete data;

[0016] The moving average filtering method is used to clean the data to reduce the noise and data fluctuations in the data. The expression is as follows:

[0017]

[0018] Among them, y t is the filtered value at time t, indicating the smoothed value at the current moment; x t-i is the input data value i time steps before time t; Q is the normalization factor, and n is the number of time steps for calculating the output;

[0019] The data is discretized using the low-pass filter transfer function H(s), which is expressed as follows:

[0020]

[0021] Where s is a complex frequency domain variable; ω c =2πf c ,ω c is the angular frequency corresponding to the cutoff frequency, f c is the cutoff frequency; r is the order of the filter;

[0022] The continuous data is divided into windows of fixed size by the window segmentation method, and the data features are extracted in the form of sliding windows;

[0023] The fast Fourier transform formula is used to extract time domain features and frequency domain features from each sliding window. The expression is as follows:

[0024]

[0025] Where X(k) is the kth component in the frequency domain; x(c) is the time domain signal; C is the number of sampling points; k is the frequency index, and k = 0, 1, 2, ..., C-1; c is the time index; e j2πkc / C is the complex exponential term; j is the imaginary unit; 2π / C is the basic angular frequency; kc is the current phase angle;

[0026] The extracted time domain features and frequency domain features are fused and standardized. The expression is as follows:

[0027]

[0028] Among them, Z is the standardized value; X is the original data value; μ is the mean of the sample data; σ is the standard deviation of the sample data.

[0029] Preferably, the motion state data includes static, vibration and displacement state data.

[0030] Preferably, the step 2 comprises:

[0031] Divide the preprocessed motion state data into a training set, a validation set, and a test set;

[0032] Use the CNN convolution layer to convolve the standardized value. The expression is as follows:

[0033]

[0034] Among them, Q i,j is the value of the output feature map at position (i, j); K m,n is the weight of the convolution kernel; I i+m,j+n is the value of the corresponding position of the input feature map; a and b are the height and width of the convolution kernel respectively;

[0035] Use the activation function f(x) to introduce nonlinear feature judgment and alleviate the gradient disappearance problem. The expression is as follows:

[0036] f(x)=max(0,x);

[0037] The state information is updated through the recursive neural network layer, and the update information content is selected. The recursive neural network layer includes: forget gate, candidate state, unit state, output gate and hidden state update. The expression is as follows:

[0038] f t =σ(W f ·[h t-1 ,x t ]+b f );

[0039] i t =σ(W i ·[ht-1 ,x t ]+b i );

[0040]

[0041] o t =σ(W o ·[h t-1 ,x t ]+b o );

[0042] h t =o t ⊙tanh(C t );

[0043] Among them, h t-1 is the hidden state at time step t-1, x t is the current input, σ() is the sigmoid function, ⊙ is the element-wise multiplication; f t is the forget gate, W f is the forget gate weight matrix, b f is the forget gate bias vector; i t is the input gate, W i is the weight matrix of the input gate, b i is the bias vector of the input gate; is a candidate memory unit, W C is the weight matrix of the candidate memory unit, b C The bias vector of the candidate memory unit, tanh() is the hyperbolic tangent function; C t is the state unit; o t is the output gate, W o is the weight matrix of the output gate, b o is the bias vector of the output gate; h t is a hidden state;

[0044] Use the fully connected layer to classify the state features and output them through the output layer;

[0045] Use loss function to judge the accuracy of the model;

[0046] The back-propagation chain rule is used to calculate the gradient image of the loss function to the parameters of each layer, optimize the model parameters, and evaluate the model performance on the validation set and test set. The expression is as follows:

[0047]

[0048] Among them, W is the weight matrix, α is the learning rate, It is the partial derivative of the loss function L with respect to the weight W, which represents the gradient of the loss function with respect to the weight;

[0049] When the model's judgment performance index reaches the estimated accuracy, the model is saved. If the accuracy does not meet the requirements, the training model will continue to be optimized.

[0050] Preferably, the step three comprises:

[0051] Deploy the trained model to the Linux environment, adjust the deep learning model according to the actual micro-Linux, and select the model parameters under the condition that the device performance allows;

[0052] Collect real-time data from IMU sensors in the deployment environment;

[0053] Preprocessing of real-time data;

[0054] Extract time domain and frequency domain feature information from preprocessed data;

[0055] Input the time domain and frequency domain feature information into the trained model prediction to obtain the current IMU motion state monitoring result of the object;

[0056] Output the IMU motion status monitoring results to the log information;

[0057] According to the IMU motion state monitoring results and combined with Beidou displacement monitoring information, the object's current final motion state information is output.

[0058] Preferably, the IMU sensor includes an accelerometer, a gyroscope, a magnetometer and a thermometer.

[0059] The application of the technical solution of the present invention has the following beneficial effects:

[0060] (1) The present invention adopts a deep learning method, utilizes the acceleration and gyroscope data provided by the IMU sensor, and combines multi-source IMU data to judge the motion state information from multiple dimensions, thereby accurately judging the device state, such as device stillness, device shaking, device displacement and displacement direction, etc. The deep learning method is used to optimize the IMU data and optimize the neural network model, making it more suitable for monitoring the motion state of the device.

[0061] (2) The present invention uses low-cost and low-power IMU equipment, which can slightly increase the power consumption of the Beidou displacement monitoring equipment while providing auxiliary status information judgment for the Beidou monitor, helping the Beidou displacement monitor to identify abnormal gross errors, and can also independently judge the equipment status information. The IMU data and Beidou displacement monitoring are integrated to improve the reliability of Beidou displacement monitoring equipment monitoring.

[0062] (3) The data processing module of this embodiment uses a micro Linux system, which is mainly used for data collection, model operation and model update. The Linux system has the function of receiving Beidou data. The data processing module has strong performance and can simultaneously meet the needs of receiving Beidou information and running the motion state deep learning algorithm.

[0063] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the aforementioned object motion state monitoring method is implemented.

[0064] The present invention also provides an electronic device, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the aforementioned object motion state monitoring method by executing the executable instructions.

[0065] In addition to the above-described objects, features and advantages, the present invention has other objects, features and advantages. The present invention will be further described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0067] Figure 1 is a flow chart of a method for monitoring the motion state of an object in an embodiment;

[0068] Figure 2 It is a schematic diagram of the deep learning model training of the embodiment. DETAILED DESCRIPTION

[0069] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0070] Example:

[0071] like Figure 1 As shown, a method for monitoring the motion state of an object comprises the following steps:

[0072] Step 1: Acquire and pre-process the motion state data of the object through the IMU sensor; the motion state data includes static, vibration and displacement state data, and the IMU sensor includes an accelerometer, a gyroscope, a magnetometer and a thermometer. Step 1 specifically includes:

[0073] Collect the motion state data of the object under test in different states; classify the motion state data by labels; and store the labeled data in a database or file system.

[0074] Data cleaning: Perform data cleaning on the collected motion state data to remove noise and incomplete data;

[0075] The moving average filtering method is used to clean the data to reduce the noise and data fluctuations in the data. The expression is as follows:

[0076]

[0077] Among them, y t is the filtered value at time t, indicating the smoothed value at the current moment; x t-i is the input data value i time steps before time t; Q is the normalization factor, and n is the number of time steps for calculating the output;

[0078] Low-pass filtering: Use the low-pass filter transfer function H(s) to discretize the data, which is expressed as follows:

[0079]

[0080] Where s is a complex frequency domain variable; ω c =2πf c ,ω c is the angular frequency corresponding to the cutoff frequency, f c is the cutoff frequency; r is the order of the filter;

[0081] Window segmentation: Continuous data is divided into windows of fixed size through the window segmentation method, and data features are extracted in the form of sliding windows;

[0082] Feature extraction: The fast Fourier transform formula is used to extract time domain features and frequency domain features from each sliding window. The expression is as follows:

[0083]

[0084] Where X(k) is the kth component in the frequency domain; x(c) is the time domain signal; C is the number of sampling points; k is the frequency index, and k = 0, 1, 2, ..., C-1; c is the time index; e j2πkc / C is the complex exponential term; j is the imaginary unit; 2π / C is the basic angular frequency; kc is the current phase angle;

[0085] Data standardization: The extracted time domain features and frequency domain features are fused and standardized. The expression is as follows:

[0086]

[0087] Among them, Z is the standardized value; X is the original data value; μ is the mean of the sample data; σ is the standard deviation of the sample data.

[0088] Step 2: Use the preprocessed motion state data to train the deep learning model, see Figure 2 This is a diagram of deep learning model training, including:

[0089] Divide the preprocessed motion state data into a training set, a validation set, and a test set;

[0090] Use the CNN convolution layer to convolve the standardized value. The expression is as follows:

[0091]

[0092] Among them, Q i,j is the value of the output feature map at position (i, j); K m,n is the weight of the convolution kernel; I i+m,j+n is the value of the corresponding position of the input feature map; a and b are the height and width of the convolution kernel respectively;

[0093] Use the activation function f(x) to introduce nonlinear feature judgment and alleviate the gradient disappearance problem. The expression is as follows:

[0094] f(x)=max(0,x);

[0095] Through the recursive neural network layer, that is, the RNN layer in the figure, the state information is updated and the update information content is selected. The recursive neural network layer includes: forget gate, candidate state, unit state, output gate and hidden state update. The expression is as follows:

[0096] f t =σ(W f ·[h t-1 ,x t ]+b f );

[0097] i t =σ(W i ·[h t-1 ,x t ]+b i );

[0098]

[0099] o t =σ(W o ·[h t-1 ,x t ]+b o );

[0100] h t =o t ⊙tanh(C t );

[0101] Among them, ht-1 is the hidden state at time step t-1, x t is the current input, σ() is the sigmoid function, ⊙ is the element-wise multiplication; f t is the forget gate, W f is the forget gate weight matrix, b f is the forget gate bias vector; i t is the input gate, W i is the weight matrix of the input gate, b i is the bias vector of the input gate; is a candidate memory unit, W C is the weight matrix of the candidate memory unit, b C The bias vector of the candidate memory unit, tanh() is the hyperbolic tangent function; C t is the state unit; o t is the output gate, W o is the weight matrix of the output gate, b o is the bias vector of the output gate; h t is a hidden state;

[0102] Use the fully connected layer to classify the state features and output them through the output layer;

[0103] Use loss function to judge the accuracy of the model;

[0104] The back-propagation chain rule is used to calculate the gradient image of the loss function to the parameters of each layer, optimize the model parameters, and evaluate the model performance on the validation set and test set. The expression is as follows:

[0105]

[0106] Among them, W is the weight matrix, α is the learning rate, It is the partial derivative of the loss function L with respect to the weight W, which represents the gradient of the loss function with respect to the weight;

[0107] When the model's judgment performance index reaches the estimated accuracy, the model is saved. If the accuracy does not meet the requirements, the training model will continue to be optimized.

[0108] Step 3: Deploy the trained deep learning model to process the data collected by the IMU sensor in real time to obtain the IMU motion state monitoring results of the object; specifically:

[0109] Deploy the trained model to the Linux environment, adjust the deep learning model according to the actual micro-Linux, and select the model parameters under the condition that the device performance allows;

[0110] Collect real-time data from IMU sensors in the deployment environment;

[0111] Preprocessing of real-time data;

[0112] Extract time domain and frequency domain feature information from preprocessed data;

[0113] Input the time domain and frequency domain feature information into the trained model prediction to obtain the current IMU motion state monitoring result of the object;

[0114] Output the IMU motion status monitoring results to the log information;

[0115] According to the IMU motion state monitoring results and combined with Beidou displacement monitoring information, the object's current final motion state information is output.

[0116] Step 4: Based on the IMU motion state monitoring results and combined with the Beidou displacement monitoring information, output the object's final motion state information.

[0117] Step 5: Input the final object motion state information into the trained deep learning model for feedback and triggering deep learning model updates.

[0118] The present invention adopts a deep learning method, utilizes the acceleration and gyroscope data provided by the IMU sensor, and combines multi-source IMU data to judge the motion state information from multiple dimensions, thereby accurately judging the device state, such as device stillness, device shaking, device displacement and displacement direction, etc. The deep learning method is used to optimize the IMU data and optimize the neural network model, making it more suitable for monitoring the motion state of the device.

[0119] The present invention uses low-cost and low-power IMU equipment, which can slightly increase the power consumption of Beidou displacement monitoring equipment, provide auxiliary status information judgment for Beidou monitor, help Beidou displacement monitor identify abnormal gross errors, and independently judge equipment status information. IMU data and Beidou displacement monitoring are integrated to improve the reliability of Beidou displacement monitoring equipment monitoring.

[0120] The data processing module of this embodiment uses a micro Linux system, which is mainly used for data collection, model operation and model update. The Linux system has the function of receiving Beidou data. The data processing module has strong performance and can simultaneously meet the needs of receiving Beidou information and the operation of the motion state deep learning algorithm.

[0121] This embodiment further discloses a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the object motion state monitoring method described above in this embodiment is implemented.

[0122] This embodiment also discloses an electronic device, including: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the object motion state monitoring method as described above in this embodiment by executing the executable instructions.

[0123] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for monitoring the motion state of an object, characterized in that: The steps include: Step 1: Obtain and pre-process the motion state data of the object through the IMU sensor; Step 2: Use the preprocessed motion state data to train the deep learning model; Step 3: Deploy the trained deep learning model to process the data collected by the IMU sensor in real time to obtain the IMU motion state monitoring results of the object; Step 4: Based on the IMU motion state monitoring results and combined with the Beidou displacement monitoring information, output the object's final motion state information.

2. The object motion state monitoring method according to claim 1, characterized in that: The method also includes step five: inputting the final object motion state information into the trained deep learning model for feedback and triggering the deep learning model update.

3. The object motion state monitoring method according to claim 2, characterized in that: The step one comprises: Collect motion state data of the object to be tested under different states; Label and classify motion status data; Perform data cleaning on the collected motion state data to remove noise and incomplete data; The moving average filtering method is used to clean the data to reduce the noise and data fluctuations in the data. The expression is as follows: Among them, y t is the filtered value at time t, indicating the smoothed value at the current moment; x t-i is the input data value i time steps before time t; Q is the normalization factor, and n is the number of time steps for calculating the output; The data is discretized using the low-pass filter transfer function H(s), which is expressed as follows: Where s is a complex frequency domain variable; ω c =2πf c ,ω c is the angular frequency corresponding to the cutoff frequency, f c is the cutoff frequency; r is the order of the filter; The continuous data is divided into windows of fixed size by the window segmentation method, and the data features are extracted in the form of sliding windows; The fast Fourier transform formula is used to extract time domain features and frequency domain features from each sliding window. The expression is as follows: Where X(k) is the kth component in the frequency domain; x(c) is the time domain signal; C is the number of sampling points; k is the frequency index, and k = 0, 1, 2, ..., C-1; c is the time index; e j2πkc / C is the complex exponential term; j is the imaginary unit; 2π / C is the basic angular frequency; kc is the current phase angle; The extracted time domain features and frequency domain features are fused and standardized. The expression is as follows: Among them, Z is the standardized value; X is the original data value; μ is the mean of the sample data; σ is the standard deviation of the sample data.

4. The object motion state monitoring method according to claim 3, characterized in that: The motion state data includes static, vibration and displacement state data.

5. The object motion state monitoring method according to claim 4, characterized in that: The second step comprises: Divide the preprocessed motion state data into a training set, a validation set, and a test set; Use the CNN convolution layer to convolve the standardized value. The expression is as follows: Among them, Q i,j is the value of the output feature map at position (i, j); K m,n is the weight of the convolution kernel; I i+m,j+n is the value of the corresponding position of the input feature map; a and b are the height and width of the convolution kernel respectively; Use the activation function f(x) to introduce nonlinear feature judgment and alleviate the gradient disappearance problem. The expression is as follows: f(x)=max(0,x); The state information is updated through the recursive neural network layer, and the update information content is selected. The recursive neural network layer includes: forget gate, candidate state, unit state, output gate and hidden state update. The expression is as follows: f t =σ(W f ·[h t-1 ,x t ]+b f ); i t =σ(W i ·[h t-1 ,x t ]+b i ); the t =σ(W o ·[h t-1 ,x t ]+b o ); h t =o t ⊙tanh(C t ); Among them, h t-1 is the hidden state at time step t-1, x t is the current input, σ() is the sigmoid function, ⊙ is the element-wise multiplication; f t is the forget gate, W f is the forget gate weight matrix, b f is the forget gate bias vector; i t is the input gate, W i is the weight matrix of the input gate, b i is the bias vector of the input gate; is a candidate memory unit, W C is the weight matrix of the candidate memory unit, b C The bias vector of the candidate memory unit, tanh() is the hyperbolic tangent function; C t is the state unit; o t is the output gate, W o is the weight matrix of the output gate, b o is the bias vector of the output gate; h t is a hidden state; Use the fully connected layer to classify the state features and output them through the output layer; Use loss function to judge the accuracy of the model; The back-propagation chain rule is used to calculate the gradient image of the loss function to the parameters of each layer, optimize the model parameters, and evaluate the model performance on the validation set and test set. The expression is as follows: Among them, W is the weight matrix, α is the learning rate, It is the partial derivative of the loss function L with respect to the weight W, which represents the gradient of the loss function with respect to the weight; When the model's judgment performance index reaches the estimated accuracy, the model is saved. If the accuracy does not meet the requirements, the training model will continue to be optimized.

6. The object motion state monitoring method according to claim 5, characterized in that: The step three comprises: Deploy the trained model to the Linux environment, adjust the deep learning model according to the actual micro-Linux, and select the model parameters under the condition that the device performance allows; Collect real-time data from IMU sensors in the deployment environment; Preprocessing of real-time data; Extract time domain and frequency domain feature information from preprocessed data; Input the time domain and frequency domain feature information into the trained model prediction to obtain the current IMU motion state monitoring result of the object; Output the IMU motion status monitoring results to the log information; According to the IMU motion state monitoring results and combined with Beidou displacement monitoring information, the object's current final motion state information is output.

7. The object motion state monitoring method according to claim 1, characterized in that: The IMU sensor includes an accelerometer, a gyroscope, a magnetometer and a thermometer.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the object motion state monitoring method according to any one of claims 1 to 7 is implemented.

9. An electronic device, characterized in that: include: processor; and a memory for storing executable instructions for the processor; Wherein, the processor is configured to execute the object motion state monitoring method as described in any one of claims 1-7 by executing the executable instructions.

Images