A self-supervised identification method for open-pit mine truck drivers across modalities

Abstract

A cross-modal self-supervised identification method for open-pit mining truck drivers belongs to the field of trajectory data mining technology. The method is as follows: GPS and OBD data of the open-pit mining trucks are acquired and preprocessed; the preprocessed GPS data is processed through a road segment semantic embedding module to obtain a road segment sequence containing spatial semantic information; based on the road segment sequence and the preprocessed OBD data, a cross-modal fusion feature extraction module, combined with self-supervision, is used to obtain a representation that showcases the unique driving style of each driver; based on the driver representation, a driver identification module is used to obtain the driver ID. This method fully considers the influence of spatial semantic information on driver operation, extracts information from both GPS and OBD data modalities to improve representation quality, thereby improving recognition accuracy; and uses self-supervised training to solve the problem of difficult model training due to limited labeled data.

Description

Technical Field

[0001] This invention relates to the field of trajectory data mining technology, and in particular to a self-supervised identification method for open-pit mine truck drivers across modalities. Background Technology

[0002] Transportation is a crucial part of the daily production process in open-pit mines. The open-pit mining environment is complex and variable, requiring strict fleet management to ensure both safety and efficiency, and assigning each driver their most suitable driving tasks. Furthermore, transportation is billed based on the number of trips. Therefore, driver identification is of paramount importance.

[0003] Existing methods for identifying open-pit mining truck drivers using other sensors primarily involve IC card identification, inputting identity information via a display screen, fingerprint recognition, or facial recognition. While these methods are effective in everyday vehicle applications, their use in open-pit mines suffers from various drawbacks due to the harsh working environment and the high cost of these high-precision sensors. To control costs and achieve effective identification, a more reasonable approach is to use vehicle trajectory data for driver identification. By utilizing the trajectory data and vehicle information provided by the GPS and OBD sensors installed in the open-pit mining trucks, the driver's driving behavior can be analyzed to extract each driver's unique driving style, thereby enabling driver identification. The GPS sensor is abbreviated as GPS (Global Positioning System); the OBD sensor is abbreviated as OBD (On-Board Diagnostics).

[0004] GPS and OBD data from open-pit mining trucks are represented as time-series sequences. For sequence-type data, recurrent convolutional network (RCN) models are typically chosen for processing. However, these models take a single-modal time-series sequence as input. Since the data in open-pit mining trucks originates from different sensors, belongs to different modalities, and has different sampling frequencies, the lengths of the GPS and OBD sequences generated during a trip are inconsistent. If timestamps are used to align the data between the two modalities and merge them into a single sequence, information represented in the sequence will be lost, and noise will be introduced. Therefore, how to use two unequal-length modal sequences as input to obtain the correlation between the two modalities and improve the representation quality of the driver's driving style remains an unsolved problem. Driving behavior is the reaction of different drivers to the external driving environment and is subject to various potential influences from complex background factors. Road conditions in open-pit mines are complex, and driving empty or fully loaded trucks significantly impacts driving operations. Therefore, analyzing driving behavior without considering semantic information in the trajectory makes it difficult to obtain accurate driving styles. Furthermore, the semantics of each road segment not only include information from the current segment but also the influence of adjacent road segments on the semantics of the current road. Therefore, embedding only information from the current road segment cannot fully express the semantics of the road segment, making it difficult to improve the accuracy of driver identification. Furthermore, most open-pit mine truck trajectory data lacks labels, making it impossible to train a model with accurate identification performance using traditional supervised learning methods. Thus, conventional driver identification methods struggle to achieve accurate identification results in open-pit mine environments. Summary of the Invention

[0005] To address the aforementioned problems, this invention provides a cross-modal self-supervised identification method for open-pit mine truck drivers. This method solves the problems of low identification accuracy caused by the difficulty in simultaneously utilizing information from both GPS and OBD data modalities when identifying drivers in open-pit mines, low identification accuracy caused by considering only driving behavior without taking into account the rich semantic information in the trajectory and the influence of external factors, and difficulty in model training due to the lack of labels.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows: a cross-modal self-supervised identification method for open-pit mine truck drivers, comprising three modules: a road segment semantic embedding module, a cross-modal fusion feature extraction module, and a driver identification module;

[0007] First, GPS trajectory data and OBD vehicle data of open-pit mining trucks are acquired, and outlier points are discarded and numerical values ​​are normalized, i.e., preprocessing. Second, a road segment semantic embedding module composed of GCN is used to semantically embed road network data and functional area information, and road network matching technology is used to transform GPS trajectory data into road segment sequences with spatial semantic information. Third, a cross-modal fusion feature extraction module is used to fuse OBD vehicle data representing driving behavior features with road segment sequences with spatial semantic information features and extract features to obtain the unique representation of the driver reflected in the trajectory. At the same time, the model is trained in a self-supervised manner. Finally, the driver representation generated by each trajectory is input into the driver identification module, and the final identification result, i.e., the driver's identity, is output.

[0008] A cross-modal self-supervised identification method for open-pit mine truck drivers includes the following steps:

[0009] Step 1: Obtain GPS trajectory data and OBD vehicle data of open-pit mining trucks and perform preprocessing: including discarding outliers in the GPS sequence and normalizing the monitoring data in the OBD sequence to obtain preprocessed GPS and OBD sequences;

[0010] Step 2: Construct a road segment map based on the open-pit mine functional area information and road network information. Then, use the road segment semantic embedding module to further encode the road segment map, mapping the points in the GPS sequence to the encoded road segment map to obtain a road segment sequence with spatial semantic information.

[0011] Step 3: Based on the OBD sequence processed in Step 1 and the road segment sequence after semantic embedding in Step 2, input them into the cross-modal fusion feature extraction module. The driving behavior information in the OBD sequence and the spatial semantic information in the road segment sequence are fused and feature extracted as the behavioral feature representation of each driver.

[0012] Step 4: Based on the behavioral feature representation of each driver in Step 3, input it into the driver identification module to obtain the identification result, i.e., the driver's identity;

[0013] Step 5: Based on the input OBD sequence and GPS sequence, construct anchor samples, positive samples and negative samples, and train the model in a self-supervised manner. In the feature space, make the anchor samples as close as possible to the positive samples and as far away as possible from the negative samples.

[0014] In step 1, the acquired GPS trajectory data and OBD vehicle data are preprocessed, including discarding outliers in the GPS sequence and normalizing the values ​​in the OBD sequence.

[0015] The method for discarding outliers in the GPS sequence is as follows:

[0016] Based on the acquired GPS trajectory data, using Let N represent the set of trajectories of driver dr, where N is the set of trajectories of driver dr. dr This represents the number of tracks generated by the driver. This represents a trajectory belonging to driver dr, where each GPS point p i =<lat,lng,alt,ts> , where represent longitude, latitude, altitude, and timestamp, respectively, and |T| represents the length of the trajectory segment. The average velocity between two consecutive GPS trajectory points is calculated based on the distance and time interval between them. If the average velocity exceeds a certain threshold, the GPS trajectory point is discarded. The calculation formula is as follows:

[0017]

[0018] Set the velocity value of the first point in the sequence to 0.

[0019] The numerical normalization method in the OBD sequence is as follows:

[0020] use This indicates that the driver (dr) generates a set of OBD columns, N. dr The number of sequences generated for this driver; This represents an OBD sequence generated by the driver (dr) during driving, where each point contains speed, acceleration, heading, clutch pedal value, accelerator pedal value, brake pedal value, engine speed, timestamp, and q. k =<speed,acc,heading,cluPedal,accPedal,brakePedal,rpm,ts> Each value represents different vehicle information, and the differences between values ​​are significant. For example, the pedal value is between 0 and 1, while the engine speed is often greater than 1000. Therefore, max-min normalization is used to normalize each feature, normalizing each feature to a value between 0 and 1. The calculation formula is as follows:

[0021]

[0022] Where speed min speed max These represent the minimum and maximum speed values ​​among all OBD data, respectively. The normalization operation for other features is the same.

[0023] In step 2, a road segment graph is constructed based on the open-pit mine road network data and functional area information, with road segments as nodes and the connectivity between road segments as edges. Then, the road segment semantic embedding module is used to further encode the road segment graph.

[0024] The functional area information is as follows: the functional areas in an open-pit mine are divided into loading areas and unloading areas, and the coverage area of ​​the functional areas is represented by latitude and longitude coordinates. This represents the i-th loading area, where 1 represents the loading area identifier and 2 represents the unloading area identifier. This functional area is represented by a latitude and longitude coordinate system with the same coordinates at both ends.

[0025] The road segment semantic embedding module is characterized in that it consists of two layers of GCN. After the road segment map is further encoded using GCN, each node in the map contains not only the information of the node itself, but also the information of some neighboring nodes, so as to better represent the semantics of the driver in this road; the GCN is Graph Convolutional Network.

[0026] The semantic embedding process for the road segment is as follows:

[0027] First, based on open-pit mine road network data and functional area information, a road segment graph is constructed with road segments as vertices and the connections between road segments as edges. The road network data is represented as a directed graph, in the form of: Each of them Indicates a road intersection. in x represents the number of points in the graph. j and y j Represent Longitude and latitude Indicates from v i to v j The directed edges connect The vertices in the graph, each edge It contains three attributes: 1) the GPS locations of the start and end points of the road segment; 2) the length of the road segment; and 3) the road type. Based on the road network data and combined with the functional area information in the open-pit mine, a road segment map with road segments as vertices is constructed. Each vertex R i Let A represent a road segment, and let A be the adjacency matrix. The expression for adjacency matrix A is:

[0028]

[0029] Among them, A ij v represents the value of the node located in the i-th row and j-th column of the matrix in the adjacency matrix. i Representing nodes i and v j Let A represent node j, E represent the edge set, and A represent the edge set. ij =1 indicates that from v i to v j There exists an edge, A ij =0 indicates that there is no edge.

[0030] X is a feature matrix. The features of each road segment are stored as vectors. The feature vectors of each road segment are concatenated in order of road segment number to generate the feature matrix X, which has the same form as A. Each row of X... i This represents all the features of a road segment, including the segment's start point, end point, length, direction, functional area type at the start point, functional area type at the end point, road level, and number of lanes. The segment's start point, end point, length, and direction undergo the same max-min normalization process as the OBD sequence. For features representing categories, such as functional area type at the start point, functional area type at the end point, road level, and number of lanes, normalization is performed by dividing the current feature value by the total number of lanes in that category.

[0031] Secondly, the road segment map The input is fed into the road segment semantic embedding module.

[0032] The first layer of the GCN takes the adjacency matrix A and the feature matrix X as input and generates a low-dimensional feature matrix, defined as:

[0033]

[0034] in It is a symmetric normalized adjacency matrix. D is the degree matrix, where each value represents the number of connections between the node and other nodes, and W0 is the learnable parameter in the GCN layer.

[0035] The second-layer GCN uses the output of the upper-layer GCN. As input, to maintain connectivity between nodes, the adjacency matrix A is not modified. The second-layer GCN is defined as follows:

[0036]

[0037] After passing through two layers of GCN, its output Where h represents the total number of road segments, g represents the feature dimension, and each row z i This represents a fixed-length feature vector with deep semantic information for each road segment node.

[0038] Secondly, based on GPS data and open-pit mine road network data, each trajectory point p... i The latitude and longitude information in the data is matched with road segments in the road network, and the matched trajectory points are represented as follows: <R i ,rt,ts>, where R i This represents the road segment ID, and rt represents the percentage of movement of this GPS point within this road segment. The calculation method is as follows:

[0039]

[0040] Where distance represents the distance between the current GPS point and the starting point of the road segment. This represents the length of the road segment, and ts represents the timestamp. Unlike conventional application scenarios, in open-pit mines, road elevation varies significantly, and elevation information is not reflected in the road network data. Therefore, the elevation data carried by each GPS point needs to be further embedded. In the GPS sequence, the elevation difference altdis between each GPS point and the previous GPS point is calculated, and the elevation difference of the first GPS point is defined as 0. Then, based on the time order of the matched trajectory points, the GPS sequence is converted into a sequence composed of points on each road segment. According to the road segment number, these road segment information are selected from the encoded road segment feature matrix Z, and the movement ratio is added. Each point in the embedded road segment sequence is... in Indicates road segment R i In the feature vector of the road segment feature matrix Z, rt represents the movement rate of the GPS point on this road segment, and altdis represents the height difference between the GPS point and the previous GPS point.

[0041] In step 3, the preprocessed OBD sequence from step 1 and the road segment sequence after spatial semantic embedding in step 2 are input into the cross-modal fusion feature extraction module to obtain a fused feature sequence. Then, feature extraction is performed to generate a final representation for each driver. This module consists of a one-dimensional temporal convolutional layer, a location embedding layer, a cross-modal attention fusion layer, and a feature extraction layer.

[0042] The cross-modal fusion feature extraction process is as follows:

[0043] Based on the preprocessed OBD sequence in step 1 Where |K| is the sequence length, 7 is the feature dimension, and the road segment sequence after semantic embedding in step 2 is... Where |T| is the sequence length, and (g+2) is the feature dimension. To address the issue of inconsistent trajectory lengths, a maximum value is assigned to each of the two sequences; sequences that do not reach the maximum length are padded with 0s at the end. The padded OBD sequence... Road segment sequence The two sequences are input into a one-dimensional temporal convolutional layer, and the feature dimensions of both sequences are transformed to d. The output is... The input is then fed into a positional embedding layer, which adds positional encoding information to each value in the sequence. The positional embedding layer does not change the shape of the input, and its output is... Finally, the input is fed into the cross-modal attention fusion layer. and The fusion is performed, and this layer consists of D cross-modal attention residual blocks, where the output of the u-th residual block is... The fused feature sequence output by the cross-modal attention fusion layer Will The input is fed into the feature extraction layer, and the output is a fixed-length feature vector representing the driver's unique driving style.

[0044] The one-dimensional temporal convolutional layer is characterized by:

[0045] Since the sampling frequencies of the two sequences are inconsistent, using one-dimensional temporal convolution can enable each node to have some awareness of its neighboring elements, while transforming the feature dimensions of both sequences into d, which facilitates the subsequent feature fusion process.

[0046] The one-dimensional temporal convolutional layer process is as follows:

[0047] Perform one-dimensional convolution operations on the two sequences respectively, for the road segment sequence. One-dimensional convolution using d convolution kernels is calculated using the following formula:

[0048]

[0049] Here, kernel represents the convolution kernel size. To maintain the sequence length, zeros in the kernel-1 column are padded before the convolution operation. The number of input channels is (g+2), and the number of output channels is d. The same convolution operation is performed on the OBD sequence, with the number of output channels also being d, but the number of input channels is 7.

[0050] The location embedding layer is characterized by:

[0051] The road segment sequence and the OBD sequence are two time-series sequences. However, timestamps are generally not processed simultaneously during data processing. Using a conventional Transformer will result in the loss of time-series information. Location embedding can preserve the time-series information of the sequence.

[0052] The process of the location embedding layer is as follows:

[0053] Based on the convolutional OBD sequence and road segment sequence, position embedding (PE) is performed to incorporate temporal information into the sequence. The formula for calculating the road segment sequence is as follows:

[0054]

[0055] in This represents the sequence after position embedding. This represents a GPS trajectory sequence with embedded semantic information. PE(·) represents the process of calculating the location code, which is specifically performed as follows: Consider it as a matrix of size |N|×d, where each value has a corresponding index. Calculate the position code based on the index using the following formula:

[0056]

[0057]

[0058] Where pos represents the row index of the data, i represents the column index, and d represents the feature dimension of each point in the sequence. After calculation, it is compared with... Add them together to complete the position encoding.

[0059] The location encoding operation for OBD sequences is the same as that for GPS sequences, and the calculation formula is as follows:

[0060]

[0061] in, This represents the sequence after position embedding.

[0062] The cross-modal attention fusion layer is characterized by:

[0063] Based on the information obtained in the above steps and The input is fed into a cross-modal attention fusion layer, which consists of D cross-modal attention residual blocks. Each residual block contains a cross-modal attention unit and a feedforward neural network unit. Taking the nth cross-modal attention residual block as an example, layer normalization is first performed on the two input sequences to ensure the stability of features in the data. Then, the cross-modal attention unit is used to fuse the OBD sequence. Information from the road segment sequence is incorporated into the road segment sequence. In this process, information from the OBD sequence is used to enhance the features of the road segment sequence. After passing through a layer normalization layer, the input is fed into a feedforward neural network unit, which then serves as the output of each cross-modal attention residual block. To prevent gradient vanishing, residual connections are also used between the cross-modal attention unit and the feedforward neural network unit in each residual block.

[0064] The process of the cross-modal attention fusion layer is as follows:

[0065] First, layer normalization is performed on the two input sequences respectively. and The two sequences are fused using cross-modal attention, and the calculation formula is as follows:

[0066]

[0067] Here, the query is defined as Key value is defined as Value defined as There are three weight matrices, (·) T This represents the matrix transpose operation, and softmax(·) represents the activation function used, which is calculated as follows:

[0068]

[0069] Due to K c A transpose operation was performed, Q τ and K c The operation between them can be viewed as calculating the inner product of each row vector in the two matrices. To prevent the inner product from becoming too large and to ensure gradient stability, a normalization operation is performed. Each value in the, except d represents the dimension of the feature; after passing through an activation function, a cross-modal module attention score matrix is ​​obtained, with a shape of |N|×|M|, where M represents the lengths of the road segment sequence and the OBD sequence, respectively. The (i, j)th item in this matrix represents the attention score of the i-th time step in the road segment sequence to the j-th time step in the OBD sequence. The value at the i-th time step in Y is V. c The weighted sum, whose weights are determined by the i-th row of the score matrix, outputs... Through the above cross-modal attention calculation, the length of Y is the same as that of the road segment sequence, but Y contains the feature space V. c The information in the middle.

[0070] To improve the model's generalization ability, a feedforward neural network unit is added after the cross-modal attention unit. Simultaneously, to maintain numerical stability during computation, layer normalization is performed before the input feedforward neural network unit. Residual connections are used between each unit to prevent gradient vanishing, which can hinder model training. Combining the above units, the calculation formula for the u-th cross-modal attention residual block is:

[0071]

[0072]

[0073] Where u = 1, ..., D represents the current calculation layer number, CM [i] (·) denotes the cross-modal attention computation process, LN(·) denotes layer normalization, and f [i] (·) indicates a feedforward neural network.

[0074] After the above operations, a fused feature sequence based on the OBD sequence and the road segment sequence is obtained. The resulting fused feature sequence is then input into the feature extraction layer to generate a final representation for each driver.

[0075] The feature extraction layer consists of a one-dimensional temporal convolutional unit and a fully connected unit. The one-dimensional temporal convolution is used to convert the feature dimension to 1, thereby reducing the feature dimension. The fully connected unit is used to convert the sequence length to S.

[0076] The feature extraction layer process is as follows: based on the fused feature sequence obtained in step 3... First, a convolution operation is performed using a temporal convolution unit with a kernel size of 1, and its output... Then, the fully connected unit sequence is mapped to a feature vector of the same length, and the calculation formula is as follows:

[0077]

[0078] Where softmax(·) is the activation function, providing a non-linear relationship for the mapping. The weight matrix has the following output:

[0079] In step 4, the driver recognition module is a... A fully connected neural network of 12 neural units Given the number of drivers to be identified, the activation function is softmax. Its calculation formula is:

[0080]

[0081] Its output is The index with the highest probability value is the corresponding driver ID.

[0082] In step 5, the model is trained using contrastive learning, a self-supervised learning approach. Anchor samples, positive samples, and negative samples are provided to the model. Within the feature space, the anchor samples are made as close as possible to the positive samples and as far apart as possible from the negative samples. The process is as follows:

[0083] A GPS sequence and OBD sequence generated by a driver during a trip are selected as anchor samples. A subsequence of this anchor sample is selected as a positive sample, and a subsequence of another driver's driving sequence is selected as a negative sample. All three samples undergo the processing in steps 1 to 3 to generate three driver behavior feature representations. To enable the model to distinguish between the representations generated by the anchor sample and the positive sample and the representations generated by the negative sample, the following loss function is defined:

[0084]

[0085] Among them, D ijIndicates sample x i With sample x j The Euclidean distance between them, where α is a hyperparameter. The goal of the ternary loss is to train the model so that... This optimization strategy allows the representations of the same driver to be close to each other, while the representations of different drivers differ significantly, thus solving the problem of difficulty in training models due to a lack of labeled data and making driver identification more accurate.

[0086] Beneficial effects: The above scheme comprises three modules: a road segment semantic embedding module, a cross-modal fusion feature extraction module, and a driver identification module. In the road segment semantic embedding module, road segment maps are constructed using road network data and functional area information. These maps are then encoded using the road segment semantic embedding module. Road network matching technology is then used to convert GPS sequences into road segment sequences with spatial semantic information, considering the impact of different driving scenarios on driver operations. The cross-modal fusion feature extraction module utilizes cross-modal attention to fuse driving behavior features from the OBD sequence with spatial semantic information features from the road segment sequence. This analyzes the driving styles of different drivers in specific driving scenarios, improving the quality of driver representations. The model is trained in a self-supervised manner, maximizing training even with limited labeled data. The driver identification module uses a fully connected layer and an activation function for classification, outputting the most probable driver ID for this trajectory. Compared to other open-pit mining truck driver identification methods, this method exhibits significant advantages in cross-modal self-supervised identification of open-pit mining truck drivers.

[0087] This invention addresses the problems of low recognition accuracy and difficulty in model training due to lack of labels when extracting trajectory features for driver identification in open-pit mines. These problems include the fact that trajectory feature extraction only considers driving behavior and does not take into account the rich semantic information in the trajectory and the influence of external factors, the inability to fully utilize information from both GPS and OBD data modalities, and the lack of labels.

[0088] Advantages: The road segment semantic embedding module uses GCN to encode the road segment map, ensuring that each node in the map contains not only information about itself but also information about some neighboring nodes. Using features with depth information, rather than manually designed features, as input to the subsequent model provides a better representation of the driver's semantics on this road. The cross-modal fusion feature extraction module utilizes cross-modal attention to process two sequences of different lengths, obtaining the correlation between the two modalities. It fully integrates information from the road segment sequence with information from the OBD sequence, improving the representation quality of the driver's driving style and thus enhancing driver recognition accuracy. A self-supervised training strategy optimizes the model, allowing for sufficient training even with limited labeled data. Attached image description:

[0089] Figure 1 This is a flowchart of the present invention.

[0090] Figure 2 This is a diagram of the driver identification structure of the present invention. Detailed Implementation

[0091] The method for identifying open-pit mine truck drivers includes the following three modules: a road segment semantic embedding module, a cross-modal fusion feature extraction module, and a driver identification module.

[0092] First, GPS trajectory data and OBD vehicle data of open-pit mining trucks are acquired, and outlier points are discarded and numerical values ​​are normalized, i.e., preprocessing. Second, a road segment semantic embedding module composed of GCN is used to semantically embed road network data and functional area information, and road network matching technology is used to transform GPS trajectory data into road segment sequences with spatial semantic information. Third, a cross-modal fusion feature extraction module is used to fuse OBD vehicle data representing driving behavior features with road segment sequences with spatial semantic information features and extract features to obtain the unique representation of the driver reflected in the trajectory. At the same time, the model is trained in a self-supervised manner. Finally, the driver representation generated by each trajectory is input into the driver identification module, and the final identification result, i.e., the driver's identity, is output.

[0093] The specific steps are as follows:

[0094] Step 1: Obtain GPS trajectory data and OBD vehicle data of open-pit mining trucks and perform preprocessing: including discarding outliers in the GPS sequence and normalizing the monitoring data in the OBD sequence to obtain preprocessed GPS and OBD sequences;

[0095] Step 2: Construct a road segment map based on the open-pit mine functional area information and road network information. Then, use the road segment semantic embedding module to further encode the road segment map, mapping the points in the GPS data to the encoded road segment map to obtain a road segment sequence with spatial semantic information.

[0096] Step 3: Based on the OBD sequence processed in Step 1 and the road segment sequence after semantic embedding in Step 2, input them into the cross-modal fusion feature extraction module. The driving behavior information in the OBD sequence and the spatial semantic information in the road segment sequence are fused and feature extracted as the behavioral feature representation of each driver.

[0097] Step 4: Based on the behavioral feature representation of each driver in Step 3, input it into the driver identification module to obtain the identification result, i.e., the driver's identity;

[0098] Step 5: Based on the input OBD sequence and GPS sequence, construct anchor samples, positive samples and negative samples, and train the model in a self-supervised manner. In the feature space, make the anchor samples as close as possible to the positive samples and as far away as possible from the negative samples.

[0099] In step 1, the acquired GPS trajectory data and OBD vehicle data are preprocessed, including discarding outliers in the GPS sequence and normalizing the values ​​in the OBD sequence.

[0100] The method for discarding outliers in the GPS sequence is as follows:

[0101] Based on the acquired GPS trajectory data, using Let N represent the set of trajectories of driver dr, where N is the set of trajectories of driver dr. dr This represents the number of tracks generated by the driver. This represents a trajectory belonging to driver dr, where each GPS point p i =<lat,lng,alt,ts> , where represent longitude, latitude, altitude, and timestamp, respectively, and |T| represents the length of the trajectory segment. The average velocity between two consecutive GPS trajectory points is calculated based on the distance and time interval between them. If the average velocity exceeds a certain threshold, the GPS trajectory point is discarded. The calculation formula is as follows:

[0102]

[0103] Set the velocity value of the first point in the sequence to 0.

[0104] The numerical normalization method in the OBD sequence is as follows:

[0105] use This indicates that the driver (dr) generates a set of OBD columns, N. dr The number of sequences generated for this driver. This represents an OBD sequence generated by the driver (dr) during driving, where each point contains speed, acceleration, heading, clutch pedal value, accelerator pedal value, brake pedal value, engine speed, timestamp, and q. k =<speed,acc,heading,cluPedal,accPedal,brakePedal,rpm,ts> Each value represents different vehicle information, and the differences between values ​​are significant. For example, the pedal value is between 0 and 1, while the engine speed is often greater than 1000. Therefore, max-min normalization is used to normalize each feature, normalizing each feature to a value between 0 and 1. The calculation formula is as follows:

[0106]

[0107] Where speed min speed max These represent the minimum and maximum speed values ​​among all OBD data, respectively. The normalization operation for other features is the same.

[0108] In step 2, a road segment graph is constructed based on the open-pit mine road network data and functional area information, with road segments as nodes and the connectivity between road segments as edges. Then, the road segment semantic embedding module is used to further encode the road segment graph.

[0109] The functional area information is as follows: the functional areas in an open-pit mine are divided into loading areas and unloading areas, and the coverage area of ​​the functional areas is represented by latitude and longitude coordinates. This represents the i-th loading area, where 1 represents the loading area identifier and 2 represents the unloading area identifier. The coverage area of ​​this functional area is represented by a set of latitude and longitude coordinates that are the same from beginning to end.

[0110] The road segment semantic embedding module is characterized in that it consists of two layers of GCN. After the road segment map is further encoded using GCN, each node in the map contains not only the information of the node itself, but also the information of some neighboring nodes, so as to better represent the semantics of the driver in this road; the GCN is Graph Convolutional Network.

[0111] The semantic embedding process for the road segment is as follows:

[0112] First, based on open-pit mine road network data and functional area information, a road segment graph is constructed with road segments as vertices and the connections between road segments as edges. The road network data is represented as a directed graph, in the form of: Each of them Indicates a road intersection. in x represents the number of points in the graph. j and y j Represent Longitude and latitude Indicates from v i to v j The directed edges connect The vertices in the graph, each edge It contains three attributes: 1) the GPS locations of the start and end points of the road segment; 2) the length of the road segment; and 3) the road type. Based on the road network data and combined with the functional area information in the open-pit mine, a road segment map with road segments as vertices is constructed. Each vertex R i Let A represent a road segment, and let A be the adjacency matrix. The expression for adjacency matrix A is:

[0113]

[0114] Among them, A ij v represents the value of the node located in the i-th row and j-th column of the matrix in the adjacency matrix. i Representing nodes i and v j Let A represent node j, E represent the edge set, and A represent the edge set. ij =1 indicates that from v i to v j There exists an edge, A ij =0 indicates that there is no edge.

[0115] X is a feature matrix. The features of each road segment are stored as vectors. The feature vectors of each road segment are concatenated in order of road segment number to generate the feature matrix X, which has the same form as A. Each row of X... i This represents all the features of a road segment, including the segment's start point, end point, length, direction, functional area type at the start point, functional area type at the end point, road level, and number of lanes. The segment's start point, end point, length, and direction undergo the same max-min normalization process as the OBD sequence. For features representing categories, such as functional area type at the start point, functional area type at the end point, road level, and number of lanes, normalization is performed by dividing the current feature value by the total number of lanes in that category.

[0116] Secondly, the road segment map The input is fed into the road segment semantic embedding module.

[0117] The first layer of the GCN takes the adjacency matrix A and the feature matrix X as input and generates a low-dimensional feature matrix, defined as:

[0118]

[0119] in It is a symmetric normalized adjacency matrix. D is the degree matrix, where each value represents the number of connections between the node and other nodes, and W0 is the learnable parameter in the GCN layer;

[0120] The second-layer GCN uses the output of the upper-layer GCN. As input, to maintain connectivity between nodes, the adjacency matrix A is not modified; the second-layer GCN is defined as follows:

[0121]

[0122] After passing through two layers of GCN, its output Where h represents the total number of road segments, g represents the feature dimension, and each row z i This represents a fixed-length feature vector with deep semantic information for each road segment node.

[0123] Secondly, based on GPS data and open-pit mine road network data, each trajectory point p... i The latitude and longitude information in the data is matched with road segments in the road network, and the matched trajectory points are represented as follows: <R i ,rt,ts>, where R i This represents the road segment ID, and rt represents the percentage of movement of this GPS point within this road segment. The calculation method is as follows:

[0124]

[0125] Where distance represents the distance between the current GPS point and the starting point of the road segment. This represents the length of the road segment, and ts represents the timestamp. Unlike conventional application scenarios, in open-pit mines, road elevation varies significantly, and elevation information is not reflected in the road network data. Therefore, the elevation data carried by each GPS point needs to be further embedded. In the GPS sequence, the elevation difference altdis between each GPS point and the previous GPS point is calculated, and the elevation difference of the first GPS point is defined as 0. Then, based on the time order of the matched trajectory points, the GPS sequence is converted into a sequence composed of points on each road segment. According to the road segment number, these road segment information are selected from the encoded road segment feature matrix Z, and the movement ratio is added. Each point in the embedded road segment sequence is... in Indicates road segment R i In the feature vector of the road segment feature matrix Z, rt represents the movement rate of the GPS point on this road segment, and altdis represents the height difference between the GPS point and the previous GPS point.

[0126] In step 3, the preprocessed OBD sequence from step 1 and the road segment sequence after spatial semantic embedding in step 2 are input into the cross-modal fusion feature extraction module to obtain a fused feature sequence. Feature extraction is then performed to generate a final representation for each driver. This module consists of a one-dimensional temporal convolutional layer, a location embedding layer, a cross-modal attention fusion layer, and a feature extraction layer.

[0127] The cross-modal fusion feature extraction process is as follows:

[0128] Based on the preprocessed OBD sequence in step 1 Where |K| is the sequence length, 7 is the feature dimension, and the road segment sequence after semantic embedding in step 2 is... Where |T| is the sequence length, and (g+2) is the feature dimension. To address the issue of inconsistent trajectory lengths, a maximum value is assigned to each of the two sequences; sequences that do not reach the maximum length are padded with 0s at the end. The padded OBD sequence... Road segment sequence The two sequences are input into a one-dimensional temporal convolutional layer, and the feature dimensions of both sequences are transformed to d. The output is... The input is then fed into a positional embedding layer, which adds positional encoding information to each value in the sequence. The positional embedding layer does not change the shape of the input, and its output is... Finally, the input is fed into the cross-modal attention fusion layer. and The fusion is performed, and this layer consists of D cross-modal attention residual blocks, where the output of the u-th residual block is... The fused feature sequence output by the cross-modal attention fusion layer Will The input is fed into the feature extraction layer, and the output is a fixed-length feature vector representing the driver's unique driving style.

[0129] The one-dimensional temporal convolutional layer is characterized by:

[0130] Since the sampling frequencies of the two sequences are inconsistent, using one-dimensional temporal convolution can enable each node to have some awareness of its neighboring elements, while transforming the feature dimensions of both sequences into d, which facilitates the subsequent feature fusion process.

[0131] The one-dimensional temporal convolutional layer process is as follows:

[0132] Perform one-dimensional convolution operations on the two sequences respectively, for the road segment sequence. One-dimensional convolution using d convolution kernels is calculated using the following formula:

[0133]

[0134] Here, kernel represents the convolution kernel size. To maintain the sequence length, zeros in the kernel-1 column are padded before the convolution operation. The number of input channels is (g+2), and the number of output channels is d. The same convolution operation is performed on the OBD sequence, with the number of output channels also being d, but the number of input channels is 7.

[0135] The location embedding layer is characterized by:

[0136] The road segment sequence and the OBD sequence are two time-series sequences. However, timestamps are generally not processed simultaneously during data processing. Using a conventional Transformer will result in the loss of time-series information. Location embedding can preserve the time-series information of the sequence.

[0137] The process of the location embedding layer is as follows:

[0138] Based on the convolutional OBD sequence and road segment sequence, position embedding (PE) is performed to incorporate temporal information into the sequence. The formula for calculating the road segment sequence is as follows:

[0139]

[0140] in This represents the sequence after position embedding. This represents a GPS trajectory sequence with embedded semantic information. PE(·) represents the process of calculating the location code, which is specifically performed as follows: Consider it as a matrix of size |N|×d, where each value has a corresponding index. Calculate the position code based on the index using the following formula:

[0141]

[0142]

[0143] Where pos represents the row index of the data, i represents the column index, and d represents the feature dimension of each point in the sequence. After calculation, it is compared with... Add them together to complete the position encoding.

[0144] The location encoding operation for OBD sequences is the same as that for GPS sequences, and the calculation formula is as follows:

[0145]

[0146] in, This represents the sequence after position embedding.

[0147] The cross-modal attention fusion layer is characterized by:

[0148] Based on the information obtained in the above steps and The input is fed into a cross-modal attention fusion layer, which consists of D cross-modal attention residual blocks. Each residual block contains a cross-modal attention unit and a feedforward neural network unit. Taking the nth cross-modal attention residual block as an example, firstly, layer normalization is performed on the two input sequences to ensure the stability of features in the data. Then, the cross-modal attention unit is used to fusion the OBD sequence. Information from the road segment sequence is incorporated into the road segment sequence. In this process, information from the OBD sequence is used to enhance the features of the road segment sequence. After passing through a layer normalization layer, the input is fed into a feedforward neural network unit, which then serves as the output of each cross-modal attention residual block. To prevent gradient vanishing, residual connections are also used between the cross-modal attention unit and the feedforward neural network unit in each residual block.

[0149] The process of the cross-modal attention fusion layer is as follows:

[0150] First, layer normalization is performed on the two input sequences respectively. and The two sequences are fused using cross-modal attention, and the calculation formula is as follows:

[0151]

[0152] Here, the query is defined as Key value is defined as Value defined as There are three weight matrices, (·) T This represents the matrix transpose, and softmax(·) represents the activation function used, which is calculated as follows:

[0153]

[0154] Due to K c A transpose operation was performed, Q τ and K c The operation between them can be viewed as calculating the inner product of each row vector in the two matrices. To prevent the inner product from becoming too large and to ensure gradient stability, a normalization operation is performed. Each value in the, except d represents the dimension of the feature. After passing through an activation function, a cross-modal attention score matrix is ​​obtained, with a shape of |N|×|M|, where M represents the lengths of the road segment sequence and the OBD sequence, respectively. The (i, j)th element in this matrix represents the attention score of the u-th time step in the road segment sequence to the j-th time step in the OBD sequence. The value at the i-th time step in Y is V. c The weighted sum, whose weights are determined by the i-th row of the score matrix, outputs... Through the above cross-modal attention calculation, the length of Y is the same as that of the road segment sequence, but Y contains the feature space V. c The information in the middle.

[0155] To improve the model's generalization ability, a feedforward neural network unit is added after the cross-modal attention unit. Simultaneously, to maintain numerical stability during computation, layer normalization is performed between the input feedforward neural network units, and residual connections are used between each unit to prevent gradient vanishing, which can hinder model training. Combining the above units, the calculation formula for the u-th cross-modal attention residual block is:

[0156]

[0157]

[0158] Where u = 1, ..., D represents the current calculation layer number, CM [i] (·) denotes the cross-modal attention computation process, LN(·) denotes layer normalization, and f [i] (·) indicates a feedforward neural network.

[0159] After the above operations, a fused feature sequence based on the OBD sequence and the road segment sequence is obtained. The resulting fused feature sequence is then input into the feature extraction layer to generate a final representation for each driver.

[0160] The feature extraction layer consists of a one-dimensional temporal convolutional unit and a fully connected unit. The one-dimensional temporal convolution is used to convert the feature dimension to 1, thereby reducing the feature dimension. The fully connected unit is used to convert the sequence length to S.

[0161] The feature extraction layer process is as follows: based on the fused feature sequence obtained in step 3... First, convolution operations are performed using a temporal convolutional layer unit with a kernel size of 1, and its output... Then, fully connected units are used to map sequences of different lengths to the same length, and the calculation formula is as follows:

[0162]

[0163] Where softmax(·) is the activation function, providing a non-linear relationship for the mapping. The weight matrix has the following output:

[0164] In step 4, the driver recognition module is a... A fully connected neural network of 12 neural units Given the number of drivers to be identified, the activation function is softmax. Its calculation formula is:

[0165]

[0166] Its output is The index with the highest probability value is the corresponding driver ID.

[0167] In step 5, the model is trained using contrastive learning, a self-supervised learning approach. Anchor samples, positive samples, and negative samples are provided to the model. Within the feature space, the anchor samples are positioned as close as possible to the positive samples and as far as possible to the negative samples. The process is as follows:

[0168] A GPS sequence and OBD sequence generated by a driver during a trip are selected as anchor samples. A subsequence of this anchor sample is selected as a positive sample, and a subsequence of another driver's driving sequence is selected as a negative sample. All three samples undergo the processing in steps 1 to 3 to generate three driver behavior feature representations. To enable the model to distinguish between the representations generated by the anchor sample and the positive sample and the representations generated by the negative sample, the following loss function is defined:

[0169]

[0170] Among them, D ij Indicates sample x i With sample x j The Euclidean distance between them, where α is a hyperparameter. The goal of the ternary loss is to train the model so that... This optimization strategy allows representations of the same driver to be close to each other, while representations of different drivers differ significantly, thus making driver identification more accurate.

[0171] Example 1:

[0172] The following is in conjunction with the appendix Figure 1-2 Specific embodiments of the present invention are described to enable those skilled in the art to understand the invention. However, it should be understood that the invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0173] A cross-modal self-supervised identification method for open-pit mining truck drivers is presented in an embodiment using a real GPS dataset and an OBD dataset from open-pit mining truck transportation. The GPS sampling frequency is approximately 30 seconds, the OBD sampling frequency is approximately 5 seconds, and the data collection period is one month. The method includes the following steps:

[0174] Step 1: Based on the obtained GPS trajectory data and OBD vehicle data of the open-pit mining trucks, preprocessing is performed, including calculating the average speed between each point in the GPS sequence and the previous point, and discarding points when the average speed is greater than 15 m / s; for the OBD sequence, max-min normalization is used to convert all features into values ​​between 0 and 1.

[0175] Step 2: Construct a road segment map based on the open-pit mine functional area information and road network information. Then, use the road segment semantic embedding module to further encode the road segment map. Finally, map the points in the GPS data onto the encoded road segment map to obtain a road segment sequence with spatial semantic information.

[0176] Step 3: Based on the OBD sequence processed in Step 1 and the road segment sequence after semantic embedding in Step 2, input them into the cross-modal fusion feature extraction module. The driving behavior information in the OBD sequence and the spatial semantic information in the road segment sequence are fused and feature extracted as the behavioral feature representation of each driver.

[0177] Step 4: Based on the behavioral feature representation of each driver in Step 3, input it into the driver identification module to obtain the identification result;

[0178] Step 5: Based on the input OBD sequence and GPS sequence, construct anchor samples, positive samples and negative samples, and train the model in a self-supervised manner. In the feature space, make the anchor samples as close as possible to the positive samples and as far away as possible from the negative samples.

[0179] Step 6: Experimental environment and hyperparameter settings:

[0180] The deep learning framework used is PyTorch 1.1.0, and the programming language is Python 3.8. All experiments were conducted on a computer equipped with an NVIDIA GeForce RTX 3070Ti, with a deep learning acceleration environment of CUDA 10.0 and cuDNN 7.0. During training, the Adam optimizer was used to optimize the model with a learning rate of 0.0001, an exponential decay rate of 0.9 for the first moment estimation, and an exponential decay rate of 0.999 for the second moment estimation; the feature g output by GCN was 32; in the cross-modal feature extraction module, the road segment sequence length N was 240, the OBD sequence length M was 1440, the number of attention residual blocks D was 4, the one-dimensional convolutional kernel size was 3, and the feature dimension d was 40; the α in the ternary loss function was 1; in the driver identification module... The value is 20. The total number of training iterations is 160K.

Claims

1. A cross-modal self-supervised open-pit truck driver identity recognition method, characterized in that The method comprises the following steps: Step 1: based on the obtained GPS trajectory data and OBD vehicle data of the open-pit truck, preprocessing is performed, including discarding the point in the GPS sequence when the average speed between each point and the previous point is greater than 15 m / s by calculating the average speed; The maximum and minimum value normalization is used for the OBD sequence to convert all features to values between 0 and 1; Step 2: according to the open-pit functional area information and road network information, a road segment graph is constructed, then the road segment graph is further encoded using a road segment semantic embedding module, and then the points in the GPS data are mapped to the encoded road segment graph to obtain a road segment sequence with spatial semantic information; Step 3: based on the OBD sequence processed in step 1 and the road segment sequence subjected to semantic embedding in step 2, the cross-modal fusion feature extraction module is inputted, the driving behavior information in the OBD sequence and the spatial semantic information in the road segment sequence are fused and feature extraction is performed, and a driver behavior feature representation is obtained; The cross-modal fusion feature extraction process is as follows: Step 4: based on the behavior feature representation of each driver in step 3, the driver identity recognition module is inputted, and a recognition result is obtained; Based on the OBD sequence pre-processed in step 1 , where is the sequence length, 7 is the feature dimension, and the road segment sequence after embedding semantics in step 2 , where is the sequence length, is the feature dimension; To solve the problem of inconsistent length of each trajectory segment, the longest value is specified for the two sequences respectively, and the sequence that does not reach the maximum length is filled with 0 at the end; The filled OBD sequence , the road segment sequence ; Input the two sequences into a one-dimensional time convolution layer respectively, and convert the feature dimensions of the two sequences to , the output of which is , ; Then input into the position embedding layer to add position encoding information to each value in the sequence, the position embedding layer does not change the input shape, and the output is , ; Finally, input into the cross-modal attention fusion layer to fuse and , this layer is composed of cross-modal attention residual blocks, where the output of the th residual block is , ; The fusion feature sequence output by the cross-modal attention fusion layer ; input into the feature extraction layer, output a fixed length feature vector representing the unique driving style of the driver; ​ Step 5: according to the input OBD sequence and GPS sequence, anchor samples, positive samples and negative samples are constructed, and the model is trained in a self-supervised manner, so that the distance between the anchor samples and the positive samples is as close as possible, and the distance between the anchor samples and the negative samples is far. In step 1, the GPS trajectory data and OBD vehicle data obtained are preprocessed, and the method comprises discarding abnormal points in the GPS sequence and normalizing values in the OBD sequence; 2. The cross-modal self-supervised open-pit truck driver identity recognition method according to claim 1, characterized in that: The abnormal point discarding method in the GPS sequence is: According to the distance between two consecutive GPS trajectory points and the time interval, the average speed between the two trajectory points is calculated, and when the average speed is greater than a certain threshold, the GPS trajectory point is discarded, and the calculation formula is: Based on the acquired GPS trajectory data, using Indicates driver The set of trajectories, in which The number of trajectory lines generated by this driver; Indicates that it belongs to the driver A trajectory, where each GPS point These represent longitude, latitude, altitude, and timestamp, respectively. Indicates the length of the trajectory segment; The speed value of the first point in the sequence is specified as 0; ， The value normalization method in the OBD sequence is:

3. The cross-modal open-pit truck driver identity self-supervised identification method according to claim 1, characterized in that: with representing the driver generating a set of OBD sequences, the number of sequences generated for the driver; representing the driver a segment of OBD sequences generated while driving, where each point contains speed, acceleration, heading, clutch pedal value, throttle pedal value, brake pedal value, engine speed, time stamp, where each value represents different vehicle information, and the difference between values is large, when the pedal value is between 0 and 1, and the engine speed is often greater than 1000, so the maximum-minimum normalization is used to normalize each feature to a value between 0 and 1, and the calculation formula is as follows: ， wherein , respectively represent the minimum and maximum values of the speed among all OBD data, and the normalization operation of other features is the same. In step 2, according to the open-pit road network data and functional area information, a road segment graph is constructed with road segments as nodes and the connection relationship between road segments as edges, and then the road segment graph is further encoded using a road segment semantic embedding module; ​ The function area information is that: the function area in the open-pit mine is divided into loading area and unloading area, and the function area coverage is expressed by longitude and latitude coordinates, , indicating the first loading area, wherein 1 represents the loading area identifier, the unloading area identifier is 2, and the function area coverage is expressed by a set of longitude and latitude coordinates with the same first and last coordinates. The road segment semantic embedding module is characterized in that the module is composed of two layers of GCN, and after the road segment graph is further encoded by using the module, each node in the graph contains not only the information of the node, but also the information of some adjacent nodes, so that the semantics of the driver in the road can be better represented; the GCN is a Graph Convolutional Network; The road segment semantic embedding process is as follows: Firstly, based on the open-pit mine road network data, the functional area information is constructed to build the road section as the vertex, the connection between the road sections as the edge, the road network data is expressed as the directed graph, and the form is wherein each of represents the road intersection, wherein represents the number of points in the graph, and respectively represent the longitude and latitude of , represents the directed edge from to , and connects the vertices in , each edge in the graph contains three attributes, which are: 1) the starting point and the ending point GPS position of the road section; 2) the length of the road section; and 3) the road type; based on the road network data, the functional area information in the open-pit mine is combined to construct the road section graph with the road section as the vertex wherein each vertex represents a road section, is the adjacency matrix, and the expression of the adjacency matrix is as follows: ， wherein, denotes the value in the adjacency matrix of the node located in the row and the column of the matrix, denotes the node , denotes the node , E denotes the edge set, denotes the existence of an edge from to , denotes the non-existence of an edge; For the feature matrix, the features of each road segment are stored in the form of a vector, and the feature vectors of each road segment are spliced in sequence of road segment number to generate a feature matrix , which is in the form of , Each row in represents all the features of a road segment, including the road segment start point, the road segment end point, the road segment length, the road segment direction, the start point functional area type, the end point functional area type, the road level, and the number of lanes; the road segment start point, the road segment end point, the road segment length, and the road segment direction are subjected to the same maximum and minimum normalization processing as the OBD sequence, and the features representing categories such as the start point functional area type, the end point functional area type, the road level, and the number of lanes are normalized by dividing the current feature value by the total number of the category; Second, the road segment graph is input into the road segment semantic embedding module, The first layer GCN takes the adjacency matrix and the feature matrix As input, a low-dimensional feature matrix is generated, defined as: ， wherein is a symmetric normalized adjacency matrix, , is a degree matrix, where each value represents the number of connections of that node to other nodes, are learnable parameters in the GCN layer; The second layer GCN uses the output of the upper layer GCN as input As input, the adjacency matrix is not changed to maintain the connectivity between nodes The second layer GCN is defined as ， After experiencing two layers of GCN, its output where denotes the total number of road segments, denotes the feature dimension, where each row denotes the fixed-length feature vector with deep semantic information of each road segment node; Secondly, based on GPS data and open-pit mine road network data, the matching between each trajectory point and road segment is completed by using the longitude and latitude information in the trajectory point , and the matched trajectory point is represented as , where represents the road segment ID, represents the moving rate of the GPS point on the road segment, and the calculation method is: ， in This indicates the distance between the current GPS point and the starting point of the road segment. Indicates the length of the road segment. This represents a timestamp; unlike conventional applications, in open-pit mines, road elevations vary significantly, and elevation information is not reflected in the road network data. Therefore, the elevation data carried by each GPS point needs to be further embedded; within the GPS sequence, the elevation difference between each GPS point and the previous GPS point is calculated. The elevation difference of the first GPS point is defined as 0; then, based on the time sequence of the matched trajectory points, the GPS sequence is converted into a sequence of points on each road segment; based on the road segment number, the encoded road segment feature matrix is ​​used... Select these road segment information and add the movement ratio; each point in the embedded road segment sequence is... ,in Indicates road segment In the road segment feature matrix The feature vector in This indicates the percentage of movement of the GPS point on this road segment. This indicates the height difference between the current GPS point and the previous GPS point.

4. The cross-modal self-supervised open-pit truck driver identity recognition method according to claim 1, characterized in that: The one-dimensional time sequence convolution layer: due to the inconsistent sampling frequencies of the two sequences, using one-dimensional time sequence convolution can make each node have some perception of its neighborhood elements, while converting the feature dimensions of the two sequences to , facilitating the subsequent feature fusion process; The one-dimensional time sequence convolution layer process is as follows: The one-dimensional convolution operation is performed on the two sequences respectively, and for the road segment sequence , one-dimensional convolution is performed using convolution kernels, and the calculation formula is: ， wherein, denotes the size of the convolution kernel, and in order to keep the sequence length unchanged, 0 is padded in front of the sequence before the convolution operation column, the number of input channels is , and the number of output channels is ; the same convolution operation is performed on the OBD sequence, with the difference being that the number of input channels is 7; The position embedding layer: the road segment sequence and the OBD sequence are two time sequence sequences, and when data is processed, the timestamp is generally not processed at the same time, and the use of a conventional Transformer will lose time sequence information, and the position embedding can keep the sequence time sequence information; The process of the position embedding layer is as follows: Based on the convolutional OBD sequence and the road segment sequence, in order to make the sequence have time sequence information, position embedding is performed, the position embedding is expressed in English as Position Embedding, and the English abbreviation is PE, and the calculation formula of the road segment sequence is: ， wherein represents the sequence after position embedding, represents the process of calculating the position encoding, which specifically operates as follows, and is regarded as a matrix of size, where each value has a corresponding subscript, and the position encoding is calculated according to the subscript, and the calculation formula is: ， ， wherein, represents the row index of the data, represents the column index, represents the feature dimension of each point in the sequence; after the calculation, it is added to to complete the position coding; The position encoding operation of the OBD sequence is the same as that of the GPS sequence, and the calculation formula is: ， wherein, represents the sequence after position embedding.

5. The cross-modal self-supervised open-pit truck driver identity recognition method according to claim 1, characterized in that: The cross-modal attention fusion layer is: Based on the above steps, the acquired and are input into a cross-modal attention fusion layer composed of cross-modal attention residual blocks, each of which contains a cross-modal attention unit and a feedforward neural network unit. In the first aspect, a method for training a cross-modal attention residual block is provided. The method comprises the following steps: First, the layer normalization operation is performed on the two input sequences respectively to ensure the stability of the features in the data. Then the information in the OBD sequence is fused into the road segment sequence using a cross-modal attention unit, which enhances the features of the road segment sequence using the information in the OBD sequence; after a layer normalization layer, the input is input into a feedforward neural network unit, and then is output as the output of each cross-modal attention residual block; in order to prevent gradient disappearance, residual connections are also used between the two units in each residual block; The process of the cross-modal attention fusion layer is as follows: First, the input two sequences are respectively subjected to layer normalization operation, according to the input sequence and , the two sequences are fused by using cross-modal attention, and the calculation formula is: ， wherein the query is defined as , the key value is defined as , and the value is defined as , , , are three weight matrices, denotes matrix transpose, denotes the activation function used, whose calculation formula is: ， Due to A transpose operation was performed. and The operation between them can be viewed as calculating the inner product of each row vector in the two matrices. To prevent the inner product from becoming too large and to ensure gradient stability, a normalization operation is performed. Each value in the, except , The dimension of the feature is denoted as ; after passing through an activation function, a cross-modal attention score matrix is ​​obtained, with the shape . , are the lengths of the road segment sequence and the OBD sequence, respectively; the first in this matrix The term represents the first item in the road segment sequence. The nth time step in the OBD sequence Attention score at each time step; The first in The value at each time step is The weighted sum, whose weights are determined by the first element in the score matrix. The line is determined, and its output is determined. Through the above cross-modal attention calculation, The length is the same as the road segment sequence, but Includes feature space Information in the middle; In order to improve the generalization ability of the model, a feedforward neural network unit is added after the cross-modal attention unit, and layer normalization is performed between the input feedforward neural network units in order to keep the numerical stability in calculation, residual connection is used between each unit to prevent the phenomenon that the model is difficult to train caused by gradient disappearance; through the above units, the calculation formula of the first cross-modal attention residual block is: ​ ， ， wherein, represents the current layer in which the computation is being performed, represents a cross-modal attention computation process, represents layer normalization, represents a feed-forward neural network; After the above operation, the fusion feature sequence based on the OBD sequence and the road segment sequence is obtained ; and then the obtained fusion feature sequence is input into a feature extraction layer to generate a final representation for each driver.

6. The cross-modal open-pit truck driver identity self-supervised identification method according to claim 1, characterized in that, The feature extraction layer: the layer is composed of a one-dimensional time sequence convolution unit and a full connection unit, one-dimensional time sequence convolution is used to convert the feature dimension into 1, the feature dimension is reduced, and the full connection unit is used to convert the sequence length into ; The process of the feature extraction layer is: based on the fusion feature sequence obtained in step 3 , first, convolution operation is performed using a time convolution layer unit with a convolution kernel size of 1, and the output , then, different lengths of sequences are mapped to the same length using a full connection unit, and the calculation formula is: ， wherein is an activation function, providing a non-linear relationship for the mapping, is a weight matrix, whose output is 7. The cross-modal open-pit truck driver identity self-supervised identification method according to claim 1, characterized in that, In step 4, the driver identification module is a neural network with one hidden layer of 100 neurons, and the activation function is softmax for the number of drivers to be identified. The calculation formula is: ， The output is where the subscript with the largest probability value is the corresponding driver ID.

8. The cross-modal open-pit truck driver identity self-supervised identification method according to claim 1, characterized in that, In step 5, the model is trained in a contrast learning manner in self-supervision, anchor samples, positive samples and negative samples are provided for the model, and in the feature space, the distance between the anchor sample and the positive sample is as close as possible, and the distance between the anchor sample and the negative sample is far; the process is as follows: selecting a GPS sequence and an OBD sequence generated by one driver in a trip as an anchor sample , selecting a sub-sequence of the anchor sample as a positive sample , selecting a sub-sequence of a driving sequence of another driver as a negative sample ; all the three samples go through the processing in steps 1 to 3 to generate three driver behavior feature representations; in order to enable the model to distinguish the representations generated by the anchor sample and the positive sample from the representation generated by the negative sample, the following loss function is defined: ， wherein, represents a sample and a sample between the samples, is a hyper-parameter; the goal of the triplet loss is to train the model such that ; this optimization strategy brings the representations of the same driver close to each other, while the representations of different drivers are far apart, making the driver identity recognition more accurate.

Images