Engineering machinery activity recognition method and system based on random convolution kernel time sequence feature transformation

Abstract

The application discloses an engineering machinery activity recognition method and system based on random convolution kernel time sequence feature transformation, generates an original data txt file after collecting time sequence data from a sensor, converts the original data txt file into a csv file after pretreatment, and then utilizes a random convolution kernel transformation algorithm to perform feature transformation on the time sequence data, and sends the transformed data into a linear classifier for classification and recognition. The application uses the method of performing feature transformation on the time sequence data by using the random convolution kernel, avoids huge calculation overheads during training of a deep convolution network, fully extracts multi-dimensional time sequence features through a large number of random convolution kernels, and improves the accuracy of engineering machinery activity recognition.

Description

Technical Field

[0001] This invention relates to the field of activity recognition and feature extraction, and more specifically, to a method and system for recognizing engineering machinery activities based on temporal feature transformation using random convolution kernels. Background Technology

[0002] Currently, the banking sector's supervision of pledged construction machinery mainly relies on manual oversight, which suffers from inefficiency, high costs, and the risk of collusion and fraud. Therefore, an automated supervision method is urgently needed.

[0003] In existing technologies, sensor-based activity recognition is expected to be applied to the automated pledge supervision of construction machinery, thereby reducing supervision costs, improving efficiency, and preventing collusion and fraud.

[0004] In existing technologies, the technique of training machine learning classification models based on time-series data from accelerometers to identify the activities of construction machinery has the advantages of low cost and wide applicability, and will become a key focus of industry research.

[0005] Chinese invention patent (CN113128568A) discloses a "method, system, device and storage medium for identifying excavator activity", which uses a random forest algorithm as a classifier. However, the machine learning method described in this patent requires manual design of feature transformation and has a low classification accuracy.

[0006] Chinese invention patent (CN109670548A) discloses a "multi-size input HAR algorithm based on improved LSTM-CNN". In order to solve the difficulties of feature extraction in traditional machine learning algorithms and the confusion caused by similar actions in classification and recognition, a convolutional neural network (CNN) is used for feature extraction and a multi-layer bidirectional LSTM network is used as a classifier. However, the CNN feature extraction method described in this patent is time-consuming and computationally complex.

[0007] In summary, there is a need to study an engineering machinery activity recognition system and method based on the temporal feature transformation method of random convolution kernel. Summary of the Invention

[0008] To address the shortcomings of existing technologies, the present invention aims to provide a method and system for identifying engineering machinery activities based on temporal feature transformation of random convolution kernels.

[0009] The present invention adopts the following technical solution:

[0010] The method for identifying engineering machinery activities based on temporal feature transformation using random convolution kernels includes: collecting time-series information on the three-axis acceleration and three-axis angular velocity of engineering machinery, generating raw data files, preprocessing them, and then inputting them into a classification model based on temporal feature transformation using random convolution kernels for classification and identification. In the classification model based on temporal feature transformation using random convolution kernels, the input acceleration and angular velocity time-series feature data are transformed by calling a random convolution kernel algorithm, and the transformed features are input into a linear classifier to obtain the classification result, thereby realizing the identification of engineering machinery activities.

[0011] In the above technical solution, the classification and recognition process achieved by the classification model based on temporal feature transformation of random convolution kernels further includes the following:

[0012] Step 1: Pre-set the hyperparameter for the number of kernel cores;

[0013] Step 2: For each kernel, randomly generate its five basic parameters: length, weight, bias, dilation, and padding.

[0014] Step 3: Each generated kernel is applied to all time series data, that is, each kernel is subjected to a sliding window dot product operation with the input time series to generate a feature map.

[0015] The calculation process involves performing a sliding window dot product between the input time series and the convolution kernel, using the formula... The calculation shows that X represents the time series of all inputs, w represents the generated kernel, and l... kernel The meaning is the length of the kernel (in time series convolution, the convolution kernel is one-dimensional, unlike the two-dimensional square convolution kernel in image convolution), where the subscript i refers to the i-th row of the time series, j is an auxiliary variable for sliding dot product addition, and d refers to the dilation size;

[0016] Step 4: Using the calculated feature map, generate two aggregated features for each kernel: the maximum value of the elements in the kernel and the proportion of positive values ​​in the kernel.

[0017] Step 5: If there are k kernels, 2k aggregated features will be generated. These transformed features are fed into a linear classifier, and the classification result is output after training.

[0018] Furthermore, in step 1, the number of kernels is usually set to the order of 10,000. The larger the number of kernels, the more favorable the generated features are for classification and the higher the accuracy. However, the computation time will also increase linearly. Experimental studies have shown that even when the number of kernels is on the order of 10,000, the computation time can be controlled to a relatively small level using the method of this invention.

[0019] Further, in step 2, specifically: the length is sampled with equal probability from the three values ​​{7, 9, 11}; the weight is randomly sampled from a Gaussian distribution with an expected value of 0 and a variance of 1; the bias is randomly sampled from a uniform distribution in [-1, 1]; the dilation is calculated using the formula d = floor(2^x), where the floor function represents rounding down, and x is randomly sampled from a uniform distribution in [0, A], where A is calculated using the formula... This calculation ensures the kernel length l kernel The length of the input time series is equal to `linput`. When generating each kernel, a random, equally probabilistic decision is made: whether to use zero padding. If padding is used, a certain amount of zero padding is appended to the beginning and end of each time series when applying the kernel, so that the "middle" element of the kernel is centered around each point in the time series, i.e., ((l kernel At position -1)×d) / 2, if there is no padding, the kernel will not prioritize the first and last positions in the time series. The kernel uses a point as the center and "focuses" on the pattern in the central region of the time series, while using padding, the kernel will also match the pattern at the beginning or end of the time series.

[0020] Furthermore, the linear classifier is either Ridge regression or logistic regression.

[0021] The present invention also provides an engineering machinery activity recognition system based on temporal feature transformation of random convolution kernel, comprising: a sensor data acquisition terminal, a host computer raw data generation module, and a cloud server model training module;

[0022] The sensor data acquisition terminal collects three-axis acceleration and three-axis angular velocity timing information and transmits the data back to the host computer.

[0023] The host computer raw data generation module receives the returned time series data, generates a raw data txt file, preprocesses the raw data txt file by deleting irrelevant information rows and columns, manually adds classification labels to the training data, converts the txt file into a csv file, and then uploads the csv file to the cloud server.

[0024] The cloud server receives the CSV file, and the cloud server model training module calls the random convolution kernel algorithm to perform feature transformation on the original acceleration and angular velocity time series features. The transformed features are then input into a linear classifier to train a classification model.

[0025] Furthermore, the system also includes an edge node data processing module and an edge node inference module. After the host computer's raw data generation module generates a raw data txt file, it can be sent to the edge node data processing module. The edge node data processing module preprocesses the raw data txt file, deleting irrelevant rows and columns. For training data, classification labels are manually added, and the txt file is converted into a CSV file. Subsequently, the CSV file is uploaded to the cloud server. After the cloud server trains the classification model, it downloads the model to the edge nodes. The edge node inference module saves the trained classification model, continuously receives newly collected data, calls the model to perform inference, and outputs the activity recognition and classification results in real time.

[0026] This invention transforms the collected triaxial acceleration and triaxial angular velocity time-series data using random convolution kernels, making the generated new features more suitable for model classification and training a linear classifier. This method avoids the huge computational overhead of training deep convolutional networks and fully extracts multi-dimensional time-series features through a large number of random convolution kernels, thereby improving the accuracy of engineering machinery activity identification. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of an example of the engineering machinery activity recognition system based on temporal feature transformation of random convolution kernel in this invention;

[0028] Figure 2 This is a flowchart illustrating the temporal feature transformation method based on random convolution kernels in this invention. Detailed Implementation

[0029] The present application will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention, and should not be construed as limiting the scope of protection of the present application.

[0030] In the identification of construction machinery activities, taking excavators as an example, identifiable mechanical activities include: idleness, movement, scooping, lowering of the robotic arm, turning left, and turning right. This invention's construction machinery activity identification method based on stochastic convolutional kernel temporal feature transformation can effectively identify these activities with very high accuracy. The method includes:

[0031] The time-series information of triaxial acceleration and triaxial angular velocity of construction machinery is collected to generate raw data files. After preprocessing, the data is input into a classification model based on random convolution kernel time-series feature transformation for classification and recognition. In the classification model based on random convolution kernel time-series feature transformation, the random convolution kernel algorithm is called to perform feature transformation on the input acceleration and angular velocity time-series feature data, and the transformed features are input into a linear classifier to obtain the classification result, thereby realizing the recognition of construction machinery activities.

[0032] like Figure 1 According to a specific example of the present invention, this method can be implemented in an engineering machinery activity recognition system based on random convolution kernel temporal feature transformation, the system including:

[0033] The system consists of a sensor data acquisition terminal, a host computer raw data generation module, an edge node data processing module, a cloud server model training module, and an edge node inference module.

[0034] The sensor data acquisition terminal collects timing information such as triaxial acceleration and triaxial angular velocity, and transmits the data back to the host computer via Bluetooth.

[0035] The host computer's raw data generation module receives the returned timing data via Bluetooth, generates a raw data txt file, and then sends the data to the edge node.

[0036] The edge node data processing module processes the original TXT data by deleting irrelevant rows and columns, manually adding category labels, converting the TXT file into a CSV file, and then uploading the CSV file to the cloud server.

[0037] The cloud server receives the CSV file, calls the random convolution kernel algorithm to perform feature transformation on the original acceleration and angular velocity time-series features, inputs the transformed features into a simple linear classifier (usually Ridge regression or logistic regression), trains the classification model, and then downloads the model to the edge nodes for inference.

[0038] Edge nodes store trained classification models, continuously receive newly collected data, call the models to perform inferences, and output the results of activity recognition and classification in real time.

[0039] The temporal feature transformation method based on random convolution kernels in this invention uses a large number of random convolution kernels to generate clustering features, and performs excellently on public datasets for activity recognition.

[0040] like Figure 2 The process of the temporal feature transformation method based on random convolution kernels includes:

[0041] Step 1: Specify the number of hyperparameters (kernels). Note that the larger the number of kernels, the better the generated features are for classification and the higher the accuracy. However, the computation time will also increase linearly. But experiments have shown that even when the number of kernels is on the order of 10,000, the computation time is kept relatively small.

[0042] Step 2: For each kernel, randomly generate its five basic parameters: length, weight, bias, dilation, and padding.

[0043] The length is sampled with equal probability from the three values ​​{7, 9, 11}; the weight is randomly sampled from a Gaussian distribution with an expected value of 0 and a variance of 1; the bias is randomly sampled from a uniform distribution in the range [-1, 1]; the dilation is calculated using the formula d = floor(2^x), where floor represents flooring down, and x is randomly sampled from a uniform distribution in the range [0, A], where A is calculated using the formula... This calculation ensures the kernel length l kernel The length of the input time series is equal to `linput`. When generating each kernel, a random, equally probabilistic decision is made: whether to use zero padding. If padding is used, a certain amount of zero padding is appended to the beginning and end of each time series when applying the kernel, so that the "middle" element of the kernel is centered around each point in the time series, i.e., ((l kernel At position -1)×d) / 2, if there is no padding, the kernel will not prioritize the first and last positions in the time series. The kernel uses a point as the center and "focuses" on the pattern in the central region of the time series, while using padding, the kernel will also match the pattern at the beginning or end of the time series.

[0044] Step 3: Each generated kernel is applied to all time-series data to produce a feature map. The calculation process involves performing a sliding window dot product between the input time series and the convolution kernel, using the formula...

[0045] Where X represents the time series of all inputs, w represents the generated kernel, and lkernel The meaning is the length of the kernel (in time series convolution, the convolution kernel is one-dimensional, unlike the two-dimensional square convolution kernel in image convolution), where the subscript i refers to the i-th row of the time series, j is an auxiliary variable for sliding dot product addition, and d refers to the dilation size;

[0046] Step 4: Using the calculated feature map, generate two aggregated features for each kernel: the maximum value of the elements in the kernel (equivalent to global max pooling) and the proportion of positive values ​​in the kernel (in this invention, this aggregated feature plays a key role in improving classification accuracy).

[0047] Step 5: If there are k kernels, 2k aggregated features will be generated. These transformed features are fed into a linear classifier, and the classification result is output after training.

[0048] As shown in the table below, taking the multi-class classification of excavator activity recognition as an example, this random convolution kernel transformation method shows a significant improvement in recognition accuracy compared to some traditional methods.

[0049]

[0050] However, those skilled in the art should understand that the above embodiments are merely preferred embodiments of the present invention, and the detailed description is only to help readers better understand the spirit of the present invention, and is not intended to limit the scope of protection of the present invention. On the contrary, any improvements or modifications made based on the inventive spirit of the present invention should fall within the scope of protection of the present invention.

Claims

1. A method for identifying engineering machinery activities based on temporal feature transformation using random convolution kernels, characterized in that, include: The time-series information of triaxial acceleration and triaxial angular velocity of construction machinery is collected, and raw data files are generated. After preprocessing, the data is input into a classification model based on random convolution kernel time-series feature transformation for classification and recognition. In the classification model based on random convolution kernel time-series feature transformation, the random convolution kernel algorithm is called to perform feature transformation on the input acceleration and angular velocity time-series feature data, and the transformed features are input into a linear classifier to obtain the classification result, thereby realizing the recognition of construction machinery activities. The classification and recognition process achieved by the classification model based on temporal feature transformation of random convolution kernels includes the following: Step 1: Pre-set the hyperparameter for the number of kernel cores; Step 2: For each kernel, randomly generate its five basic parameters: length, weight, bias, dilation, and padding. Step 3: Each generated kernel is applied to all time series data, that is, each kernel is subjected to a sliding window dot product operation with the input time series to generate a feature map. Step 4: Using the calculated feature map, generate two aggregated features for each kernel: the maximum value of the elements in the kernel and the proportion of positive values ​​in the kernel. Step 5: If there are k kernels, 2k aggregated features will be generated. These transformed features are fed into a linear classifier, and the classification result is output after training.

2. The method for identifying engineering machinery activities based on temporal feature transformation of random convolution kernels according to claim 1, characterized in that, In step 1, the number of kernel cores is set to be on the order of 10,000.

3. The method for identifying engineering machinery activities based on temporal feature transformation of random convolution kernels according to claim 1, characterized in that, In step 2, specifically: the length is sampled with equal probability from the three values ​​{7, 9, 11}; the weight is randomly sampled from a Gaussian distribution with an expected value of 0 and a variance of 1; the bias is randomly sampled from a uniform distribution in [-1, 1]; the dilation is calculated using the formula d=floor(2^x), where the floor function represents rounding down, and x is randomly sampled from a uniform distribution in [0, A], where A is calculated using the formula... This calculation ensures the kernel length l kernel The length of the input time series is equal to `linput`. When generating each kernel, a random, equally probabilistic decision is made regarding whether to use zero padding. If padding is used, a certain amount of zero padding is appended to the beginning and end of each time series when applying the kernel, so that the "middle" elements of the kernel are centered around each point in the time series. The location.

4. The method for identifying engineering machinery activities based on temporal feature transformation of random convolution kernels according to claim 1, characterized in that, The linear classifier mentioned is either Ridge regression or Logistic regression.

5. A system for recognizing engineering machinery activities based on temporal feature transformation of random convolution kernels, characterized in that, include: Sensor data acquisition terminal, host computer raw data generation module, cloud server model training module; The sensor data acquisition terminal collects three-axis acceleration and three-axis angular velocity timing information and transmits the data back to the host computer. The host computer raw data generation module receives the returned time series data, generates a raw data txt file, preprocesses the raw data txt file by deleting irrelevant information rows and columns, manually adds classification labels to the training data, converts the txt file into a csv file, and then uploads the csv file to the cloud server. The cloud server receives the CSV file, and the cloud server model training module calls the random convolution kernel algorithm to perform feature transformation on the original acceleration and angular velocity time-series features. The transformed features are then input into a linear classifier to train a classification model.

6. The engineering machinery activity recognition system based on random convolution kernel temporal feature transformation according to claim 5, characterized in that, The system also includes an edge node data processing module and an edge node inference module. After the host computer's raw data generation module generates a raw data txt file, it can be sent to the edge node data processing module. The edge node data processing module preprocesses the raw data txt file, deleting irrelevant rows and columns. For training data, classification labels are manually added, and the txt file is converted into a CSV file. The CSV file is then uploaded to the cloud server. After the cloud server trains the classification model, it downloads the model to the edge nodes. The edge node inference module saves the trained classification model, continuously receives newly collected data, calls the model to perform inference, and outputs the activity recognition and classification results in real time.

Images