Vehicle prediction method and device

Abstract

The present application discloses a vehicle prediction method and device. The vehicle prediction method includes: constructing a camera point map based on the relationship between multiple camera points and nodes; based on the camera point map, obtaining the historical records of vehicles being captured in the monitoring system to form multiple camera point sequences; constructing a vectorized model training set based on the multiple camera point sequences, and training to form a vectorized model based on the vectorized model training set; performing feature vectorization on each camera point in the camera point map based on the vectorized model to obtain multiple point feature vectors; constructing a point prediction model training set based on the multiple point feature vectors, and training to form a point prediction model based on the point prediction model training set; predicting the camera point where the vehicle will appear at the next moment based on the point prediction model. The vehicle prediction method avoids the use of manual analysis methods, does not miss the camera points where the vehicle may appear, and can be used in practical application environments such as vehicle query and parking anomaly detection.

Description

Technical Field

[0001] The present application relates to the field of vehicle monitoring technology, and in particular to a vehicle prediction method and device. Background Art

[0002] With the increasing number of vehicles in cities, traffic monitoring systems have become a major player in vehicle management. Vehicle retrieval technology plays a crucial role in this management. As the number of camera locations and the scope of management expand, the cost of retrieval increases over large areas. To reduce vehicle retrieval costs, we first identify camera locations where vehicles are likely to appear, then conduct detailed inspections within this smaller area.

[0003] Traditional search methods rely on manual analysis to select a small number of camera locations within a local area based on historical vehicle behavior, reducing vehicle search costs. However, due to manual analysis, these traditional search methods often miss locations where vehicles may be present, resulting in low detection efficiency and accuracy, making it impossible to accurately locate the target vehicle.

[0004] Application Contents

[0005] The purpose of this application is to solve the technical problem that the traditional search method using manual analysis results in low detection efficiency and low accuracy, and cannot accurately find the target vehicle. To achieve the above purpose, this application provides a vehicle prediction method and device.

[0006] This application provides a vehicle prediction method, comprising:

[0007] Acquire multiple camera points of the monitoring system and node relationships between the multiple camera points, and construct a camera point map based on the multiple camera points and the node relationships;

[0008] Based on the camera point map, a historical record of vehicles being photographed in the monitoring system is obtained to form multiple camera point sequences;

[0009] Constructing a vectorized model training set based on the multiple camera point sequences, training a vectorized model based on the vectorized model training set, and performing feature vectorization on each camera point in the camera point map based on the vectorized model to obtain multiple point feature vectors;

[0010] Constructing a point prediction model training set based on the plurality of point feature vectors, and training a point prediction model based on the point prediction model training set;

[0011] The camera position where the vehicle will appear at the next moment is predicted based on the position prediction model.

[0012] The present application provides a vehicle prediction device, comprising:

[0013] A camera point map generation module is used to obtain a plurality of camera points of the monitoring system and node relationships between the plurality of camera points, and to construct a camera point map based on the plurality of camera points and the node relationships;

[0014] A camera point sequence acquisition module is used to acquire the historical records of vehicles being photographed in the monitoring system based on the camera point map to form multiple camera point sequences;

[0015] a vectorization model generation module, configured to construct a vectorization model training set based on the plurality of camera point sequences, train a vectorization model based on the vectorization model training set, and perform feature vectorization on each camera point in the camera point map based on the vectorization model to obtain a plurality of point feature vectors;

[0016] A point prediction model generation module is used to construct a point prediction model training set based on the multiple point feature vectors, and train a point prediction model based on the point prediction model training set;

[0017] The prediction module is used to predict the camera point where the vehicle will appear at the next moment based on the point prediction model.

[0018] In the above-mentioned vehicle prediction method and device, a training set is formed by forming multiple camera point sequences to train a vectorized model. Based on the vectorized model, each camera point is characterized to form multiple point feature vectors. The feature vectors of the historical first t point sequences of multiple vehicles are used as the input of the point prediction model, and the t+1th camera point is output as the prediction result to form a training set for point prediction, so as to train and form a point prediction model. Based on the trained point prediction model, the historical point capture sequence of the vehicle under test appearing in the monitoring network is obtained to form a camera point sequence. After vectorization, the point feature vector corresponding to each historical camera point is output, and the multiple point feature vectors formed are input into the point prediction model to output the camera point where the vehicle will appear at the next moment.

[0019] Furthermore, the vehicle prediction method provided by the present application establishes the relationship between all camera points from the perspective of the entire monitoring system, covering the camera points of the entire monitoring network. And through specific data characterization of the information of the camera points of the monitoring network and the relationship between each camera point, the relationship between all camera points is fully considered. The vehicle prediction method provided by the present application is based on analyzing the historical records of vehicles appearing at each camera point, and adopts deep learning training and analysis of large-scale data to predict the camera point where the vehicle may appear at the next moment. Therefore, the vehicle prediction method provided by the present application can improve the accuracy of vehicle prediction, can be applied to large-scale vehicle monitoring systems, and can achieve accurate search of target vehicles. Therefore, the vehicle prediction method provided by the present application avoids the use of manual analysis methods, will not miss the camera points where the vehicle may appear, and can be used in practical application environments such as vehicle query and parking anomaly detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a flowchart of the steps of the vehicle prediction method provided by this application.

[0021] Figure 2 It is a structural diagram of the camera point map provided in this application.

[0022] Figure 3 This is a data table diagram of a skip-word model training set in an embodiment provided in this application.

[0023] Figure 4 It is a schematic diagram of the skip-word model structure in an embodiment provided in this application.

[0024] Figure 5 This is a schematic diagram of the vehicle prediction process in an embodiment provided by this application.

[0025] Figure 6 This is a data table diagram of a point prediction model training set in an embodiment provided in this application.

[0026] Figure 7 It is a structural diagram of the vehicle prediction device provided by this application. DETAILED DESCRIPTION

[0027] The technical solution of the present application is further described in detail below through the accompanying drawings and examples.

[0028] See Figure 1 , the present application provides a vehicle prediction method, comprising:

[0029] S10, obtaining node relationships between multiple camera points and multiple camera points of the monitoring system, and constructing a camera point map based on the multiple camera points and node relationships;

[0030] S20. Based on the camera location map, obtain the historical records of vehicle captures in the monitoring system to form multiple camera location sequences;

[0031] S30. According to the multiple camera location sequences, construct a vectorization model training set. Based on the vectorization model training set, train to form a vectorization model, and according to the vectorization model, perform feature vectorization on each camera location in the camera location map to obtain corresponding multiple location feature vectors;

[0032] S40. According to the multiple location feature vectors, construct a location prediction model training set, and based on the location prediction model training set, train to form a location prediction model;

[0033] S50. According to the location prediction model, predict the camera location where the vehicle will appear at the next moment.

[0034] In S10, the monitoring system includes v0, v1,..., v n-1 A total of n camera locations, forming n nodes of the camera location map. The sequence of being captured between two camera locations of a vehicle is used as the edge relationship between the two nodes of the camera location map, which can also be understood as the node relationship. For example: The vehicle is first captured at the v0 camera location and then at the v4 camera location. (v0, v4) forms the edge relationship between the two nodes v0 and v4. By analogy, multiple camera locations form multiple nodes of the camera location map, and the sequence of being captured between camera locations forms the node relationship of the camera location map, forming a camera location map, as Figure 2 shown.

[0035] In S20, in the urban dynamic and static traffic system, each vehicle continuously runs between each camera location, making the edge relationship in the camera location map, which can also be understood as the node relationship, dynamically changing. From the historical records of vehicle captures at each camera location generated during the system operation, a camera location map can be formed. According to the camera location map, the camera location records passed by the vehicle within a certain time period can be obtained. According to the order of vehicle appearances at the camera locations, a camera location sequence corresponding to the vehicle is formed. For example: The camera location sequence corresponding to vehicle Beijing X***4 is {v 21 , v 45 , v 111 , v 73}, the camera location sequence corresponding to Beijing B***T is {v 41 , v 75 , v 431 , v 73 , v 56}, and the camera location sequence corresponding to Beijing C***2 is {v2, v7, v 534 , v 33,v 96}.

[0036] In S30, a training set for a vectorized model is constructed using camera point sequences of multiple vehicles. The vectorized model is trained based on the training set to vectorize each camera point, thereby obtaining a feature vector corresponding to each point in the image.

[0037] Based on the trained vectorization model, feature vectorization is performed on each camera point in the camera point map to obtain the point feature vector corresponding to each camera point in the camera point map. One camera point corresponds to one point feature vector.

[0038] In S40, after the plurality of point feature vectors, the point sequences of the plurality of vehicles are constructed to form a training set for the point prediction model. The point prediction model is trained based on the training set to predict the camera point where the vehicle will appear at the next moment.

[0039] In S50, steps S10 through S50 are followed to obtain a vectorization model and a point prediction model. The vectorization model is used to perform feature vectorization on each camera point in the camera point map, obtaining a feature vectorization corresponding to each camera point. The vehicle's camera point sequence is obtained, and the multiple feature vectors corresponding to the camera point sequence are input into the point prediction model. The vehicle's camera point at the next moment is predicted, and the prediction result is output.

[0040] Through S10 to S40, multiple camera point sequences are formed to form a training set, and training is used to form a vectorized model. After obtaining the point feature vector, each camera point in the previous t-1 time period is characterized to form a sequence represented by multiple point feature vectors. It can also be understood that a camera point sequence is obtained in the previous t-1 time period, and each camera point sequence corresponds to multiple camera points, and each camera point corresponds to a point feature vector. A camera point sequence corresponds to multiple point feature vectors. The camera point sequence represents the order in which vehicles appear at the camera points within a certain time period, so that each camera point in the camera point sequence is connected to each other. Through S40, the sequence formed by the multiple point feature vectors is used as the input of the point prediction model, and the camera point at time t is output as the prediction result. The point prediction model is trained using a supervised training method. Through S50, based on the trained point prediction model, the historical point capture sequence of a vehicle appearing in the monitoring network is obtained to form a camera point sequence, the point sequence is vectorized and input into the point prediction model, and the camera point where the vehicle will appear at the next moment is output.

[0041] Furthermore, the vehicle prediction method provided by the present application establishes the relationship between all camera points from the perspective of the entire monitoring system, covering the camera points of the entire monitoring network. And through specific data characterization of the information of the camera points of the monitoring network and the relationship between each camera point, the relationship between all camera points is fully considered. The vehicle prediction method provided by the present application is based on analyzing the historical records of vehicles appearing at each camera point, and adopts deep learning training and analysis of large-scale data to predict the camera point where the vehicle may appear at the next moment. Therefore, the vehicle prediction method provided by the present application can improve the accuracy of vehicle prediction, can be applied to large-scale vehicle monitoring systems, and can achieve accurate search of target vehicles. Therefore, the vehicle prediction method provided by the present application avoids the use of manual analysis methods, will not miss the camera points where the vehicle may appear, and can be used in practical application environments such as vehicle query and parking anomaly detection.

[0042] In one embodiment, S10, obtaining node relationships between multiple camera points and multiple camera points of a monitoring system, and constructing a camera point map based on the multiple camera points and node relationships, includes:

[0043] S110, acquiring multiple camera points to form a camera point set V, and treating each camera point as a node;

[0044] S120, obtaining the order in which multiple vehicles are captured between multiple nodes, forming a node relationship between each adjacent node, and forming a node edge set E based on the node relationship;

[0045] S130 , constructing a camera point graph G(V, E) according to the camera point set V and the node edge set E.

[0046] In S110, a camera point is considered a node in the camera point graph. Multiple camera points form multiple nodes in the camera point graph. In S120, the order in which a vehicle is captured between two camera points is considered an edge relationship between two adjacent nodes, also known as a node relationship, forming a node-edge set, also known as an edge set. The node-edge set E represents the set of edges between all adjacent nodes.

[0047] In one embodiment, forming Figure 2 The camera point graph G(V,E) is shown. Taking this as an example, the camera point set V=[v0,v1,...,v n-1 ], the node edge set is E=[(v0,v1),(v0,v4),(v1,v2),…,(v n-2 ,v n-1)]. In the urban dynamic or static traffic system, as each vehicle continuously moves between various camera positions, the connection of the edges in the camera position map G(V, E) formed thereby is dynamically changing. It can also be understood that the node relationship between the various nodes in the camera position map G(V, E) is dynamically changing.

[0048] Through S110 and S130, and based on the historical data of each vehicle moving between various camera positions, a camera position map G(V, E) is formed. In the network with surveillance cameras as nodes, through the historical records of vehicles being captured before various camera positions, the possible camera positions where the vehicle may appear at the next moment can be predicted, which can be used in practical applications such as large-scale vehicle retrieval, vehicle behavior analysis, and parking anomaly detection.

[0049] In one embodiment, S20, according to the camera position map, obtain multiple camera position sequences, including:

[0050] S210, according to the camera position map G(V, E), obtain the camera position sequence corresponding to the vehicle;

[0051] Among them, the camera position sequence includes the records of the same vehicle being captured between multiple camera positions within a time period.

[0052] In S210, the camera position sequence may include the historical records of the same vehicle being captured between multiple camera positions within a time period of 3 to 5 days. For example: the camera position sequence corresponding to Beijing X***4 is {v 21 , v 45 , v 111 , v 73}, the camera position sequence corresponding to Beijing B***T is {v 41 , v​​​​​​​​​​​​​​​​Multiple camera position sequences form a vectorized model training set. The vectorized model training set includes historical records captured by vehicles at each camera position during the operation of the monitoring system. It can be understood that the camera position records passed by a vehicle within a certain time period form a camera position sequence corresponding to the vehicle in the order of the vehicle's appearance at each camera position.

[0054] In one embodiment, historical records of the same vehicle captured at each camera position within 3 days are obtained to form a camera position sequence. By selecting the time period, it is possible to avoid listing records of vehicle captures separated by a long time period in the same sequence. By selecting the time period, the complexity of vehicle prediction can be reduced. The specific time period selection can be defined according to the actual situation.

[0055] In one embodiment, in the monitoring system, using the license plate as the single identifier of the vehicle, a sequence formed by the license plate and the corresponding camera position labels passed through successively is obtained. The camera position sequences of multiple vehicles constitute the training data sets for the vectorized model and the position prediction model. For example: the camera position sequence corresponding to Beijing X***4 is {v 21 ,v 45 ,v 111 ,v 73}, the camera position sequence corresponding to Beijing B***T is {v 41 ,v 75 ,v 431 ,v 73 ,v 56}, the camera position sequence corresponding to Beijing C***2 is {v2, v7, v 534 ,v 33 ,v 96}, and so on.

[0056] In one embodiment, S30, according to multiple camera position sequences, construct a vectorized model training set, and train a vectorized model according to the vectorized model training set, including: [[ID=`35]]

[0057] S310, set the window size for training the skip-gram model;

[0058] S320, according to the window size, extract samples from each camera position sequence to form multiple groups of model training samples; where each group of model training samples includes a single camera position and the corresponding predicted context position sequence;

[0059] S330, train a skip-gram model according to multiple groups of model training samples.

[0060] In this embodiment, the vectorization model is a skip-gram model. In this embodiment, the sequence of camera positions corresponding to the vehicle is analogous to a sentence, with each camera position analogous to a word. Through modeling and training with the skip-gram model, a vector representation of each camera position is obtained, i.e., a position feature vector.

[0061] In S310, the window size of the word-skipping model can be 4, 5, 6, etc., and can be limited according to actual conditions. In this embodiment, the window size of the word-skipping model is selected as 4.

[0062] In S320, each camera position corresponds to a position feature vector. The skip-gram model predicts the context of the text using intermediate words, thereby training to obtain a vector representation for each word. In this embodiment, the vehicle's camera position sequence is analogized to a sentence, with each camera position analogized to a word. By setting the window size, the position feature vector corresponding to each camera position sequence is extracted. In S330, the multiple input sample and output sample pairs are combined to form a training set, and the skip-gram model is trained to achieve vectorization of the camera positions.

[0063] In one embodiment, taking the window size as 4 as an example, samples are extracted from each camera point sequence to form input samples and output samples. Each set of model training samples includes input samples and output samples, and the format is as follows: Figure 3 As shown in the table.

[0064] In one embodiment, S330, training a skip-word model based on multiple sets of model training samples includes:

[0065] S331, inputting a single camera point into a word-skipping model and outputting a predicted context point sequence; wherein the word-skipping model is a three-layer neural network;

[0066] S332: Training a word skipping model based on a single camera point and a sequence of predicted context points, and constructing a target loss function for the word skipping model. During training, the parameters of each layer of the model are iteratively adjusted using a gradient descent algorithm to obtain a trained word skipping model.

[0067] Among them, the target loss function of the skip-word model is:

[0068]

[0069]

[0070] v in Represents a single camera point of the input word skipping model, (v out,1 ,v out,2 ,...,v out,t ) represents the output of the word-skipping model vin The predicted context point sequence corresponding to a single camera point, t represents the window size, h represents the vector of dimension d in the hidden layer, W out Represents the d×n dimensional connection weights of the hidden layer. The predicted context point sequence can also be understood as a single camera point passing through the skip-gram model to output the front and rear camera points that the vehicle has passed.

[0071] In S331, a word-skipping model is trained based on multiple sets of model training samples. The word-skipping model is a three-layer neural network. The input layer receives a single camera point, and the output layer is the context point of the predicted sequence. In S332, the target loss function of the word-skipping model is constructed on the output layer.

[0072] See Figure 4 In S331 and S332, the input layer of the word-skipping model is a camera point v in The output layer of the word-hopping model is to predict v in the camera point sequence in The front and rear camera points, that is, {v out,1 ,v out,2 ,v out,3 ,v out,4}. {v out,1 ,v out,2 ,v out,3 ,v out,4} is the point feature vector. For example: when the camera point sequence is {v 41 ,v 75 ,v 431 ,v 73 ,v 56}, the corresponding input layer is {v 431}→The output layer is {v 41 ,v 75 ,v 73 ,v 56}. It can also be understood as v in For {v 431},{v out,1 ,v out,2 ,v out,3 ,v out,4} is {v 41 ,v 75 ,v 73 ,v 56}. Similarly, each camera point will correspond to a point feature vector.

[0073] In this embodiment, a camera point v is represented by a unique hot encoding. in The corresponding input vector has a length of n. Correspondingly, the length of the n-length unique encoding output vector represents the output v of the front and rear camera points respectively. out,1 、v out,2、v out,3 and v out,4 .like Figure 4 As shown, in one embodiment, for a certain training sample, the input layer v in The corresponding one-hot encoding input vector is 01000…0, and the output layer v out,1 The corresponding one-hot encoding is 1000…0, and the output layer v out,2 The corresponding one-hot encoding is 0001…0, and the output layer v out,3 The corresponding one-hot encoding is 0010…0, and the output layer v out,4 The corresponding one-hot encoding is 0000…1.

[0074] The weight matrix W between the input layer and the hidden layer in The kth row of represents the weight of the kth camera point. The hidden layer is a vector h of dimension d. The vector h is obtained by the weighted sum of the input layer input, and no activation function is required. The vector corresponding to each camera point output is shared with the d×n dimension connection weight of the hidden layer, which is W out The input of the output node of the word-skipping model is calculated by the weighted sum of the corresponding input nodes, that is, Where h = W in *v cj .u cj represents the cth output camera point among t output camera points. The probability of identifying it as the jth camera point among all n camera points. Finally, the probability of the cth output camera point being the jth camera point among all n camera points generated by the softmax function is:

[0075]

[0076] By using multiple sets of model training samples as the training set of the word-skipping model, the target loss function is adjusted. The target loss function of the word-skipping model is the logarithmic form of the conditional probability of the output before and after points, as shown in the formula shown.

[0077] In step S332, during the training of the point prediction model, the target loss function of the word-skipping model will continue to decrease and eventually reach a balanced state. When the target loss function reaches a balanced state, the word-skipping model reaches an optimal state, and a trained word-skipping model is obtained.

[0078] In one embodiment, when training the word skipping model, the stochastic gradient descent algorithm is used to adjust the target loss function, and negative sampling is used to reduce the computational complexity. Finally, the weight matrix W is obtained. in Each row of is used as the point feature vector of the corresponding camera point.

[0079] In one embodiment, the vectorization model may also be a CBOW model or a Glove model, etc., as long as it can realize the vectorization of the camera point.

[0080] In one embodiment, feature vectorization is performed on each camera point in the camera point map based on the vectorization model to obtain multiple corresponding point feature vectors. Alternatively, the vectorization model can be used to generate corresponding point feature vectors for each camera point after the model is applied to the preceding and following camera points that the vehicle has passed through.

[0081] See Figure 5 In one embodiment, S40 constructs a point prediction model training set based on multiple point feature vectors, and trains a point prediction model based on the point prediction model training set, including:

[0082] S410, inputting multiple point feature vectors into an attention mechanism graph convolutional network, outputting multiple node hidden layer features, and inputting the multiple node hidden layer features into a classification layer, outputting a camera point prediction result;

[0083] S420: Train the attention mechanism graph convolutional network based on multiple point feature vectors and camera point prediction results, obtain parameters of the attention mechanism graph convolutional network, and obtain a trained point prediction model.

[0084] In one embodiment, S410 inputs multiple point feature vectors into an attention mechanism graph convolutional network, outputs multiple node hidden layer features, and inputs the multiple node hidden layer features into a classification layer, outputs camera point prediction results, including:

[0085] S411, construct the attention coefficient based on the graph convolutional network, feedforward neural network and leaky rectified linear unit function; the attention coefficient is:

[0086]

[0087] in, represents a feedforward neural network, W represents a shared weight matrix, Represents the node hidden layer features corresponding to node i output by the l-th layer graph convolution, represents the node hidden layer feature corresponding to node j output by the l-th layer graph convolution, the || symbol represents vector concatenation, and nodes i and j are adjacent nodes;

[0088] S412, according to the attention coefficient, obtain the node hidden layer feature corresponding to the node i output by the l+1th graph convolution layer of the multi-head attention mechanism; the node hidden layer feature corresponding to the node i output by the l+1th graph convolution layer of the multi-head attention mechanism is:

[0089]

[0090]

[0091] Among them, σ(·) represents the activation function, N i represents the set of nodes in the neighborhood of node i, K represents the number of multi-head attention, Represents the node hidden layer features corresponding to node i output by the l+1th graph convolutional layer of the single-head attention mechanism.

[0092] S413, constructing a multi-layer graph convolution layer according to the node hidden layer features corresponding to node i output by the l+1th graph convolution layer of the multi-head attention mechanism;

[0093] S414, performing maximum pooling and average pooling on the node hidden layer features output by the multi-layer graph convolution layer to obtain two-pooled output features;

[0094] S415: Input the two pooled output features into the recognition function, and output the probability of the vehicle appearing at each camera point at the next moment;

[0095] S416 , obtaining the camera location where the vehicle will appear at the next moment based on the probability that the vehicle will appear at each camera location at the next moment.

[0096] In S411 to S416, a point prediction model training set is constructed based on multiple point feature vectors. Figure 6 As shown in the table below. The feature representation of the point sequence for each camera point sequence in the first t-1 time period is used as the input to the point prediction model, and the camera point at time t is output as the prediction result. The point prediction model in this embodiment is composed of an attention mechanism graph convolutional network, a feedforward neural network, a leaky rectified linear unit function, a dual pooling process of maximum pooling and average pooling, and a softmax function.

[0097] The order in which vehicles appear at each camera point serves as the edge relationship in the camera point graph. As each vehicle travels, the edge relationships (also understood as node relationships) change dynamically. By employing an attention-based graph convolutional network, this approach is adaptable to dynamic vehicle changes. The feature representations of the point sequences for the first t-1 time periods of multiple camera point sequences are used to form multiple point feature vectors, which serve as the input to the point prediction model. The camera point corresponding to time t is used as the prediction output of the point prediction model, which is trained using supervised training methods to form the point prediction model.

[0098] The attention mechanism graph convolutional network GAT is an improvement based on the graph convolutional network GCN by adding an attention mechanism. The relationship between each node on the graph structure is characterized by a learnable attention method. According to the characteristics of the current node and the nodes in the neighborhood (which can also be understood as adjacent nodes), a dynamic connection relationship is established. Applied in this embodiment, the order of appearance of vehicles in the camera point map is used as the relationship between the edges in the graph. As each vehicle travels, the relationship between the edges changes dynamically. In the vehicle prediction method provided in this application, the attention mechanism is used to dynamically calculate the relationship between adjacent points in the camera point map.

[0099] The camera point map composed of the historical camera point sequence of the vehicle is vectorized and converted into x=[h0,h1,...,h i ,...,h z ] form. Among them, h i Represents the feature vector of the i-th camera point in the camera point sequence of length z of the vehicle. The output of the l-th layer graph convolution calculation is the updated hidden feature H of each camera point node (l) =[h0 (l) ,h1 (l) ,...,h i (l) ,...,h t (l) All nodes train a shared weight matrix to obtain the weight of each neighbor node. The shared weight matrix is ​​the mapping relationship between the input F feature dimension and the output F' dimension.

[0100] In S411, when calculating the attention coefficient, the feature vectors of node i and node j are mapped using W respectively, and the vectors are concatenated. The concatenated vector is mapped to a real number. It is activated by the leaky rectified linear unit function LeakyReLU and normalized to obtain the final attention coefficient.

[0101] In S412, N i It represents the set of nodes in the neighborhood of node i, which can also be understood as the set of adjacent camera points of a certain camera point i, or the set of adjacent nodes of node i. According to the attention coefficient, the weighted sum of the nodes in the neighborhood of node i can be used to obtain the output feature of node i, which is: the hidden layer feature of the node corresponding to node i output by the l+1th layer of the graph convolution layer of the single-head attention mechanism Among them, α ij It is obtained using the softmax function.

[0102] In this embodiment, a multi-head attention mechanism is introduced to improve the representation ability of the model, so that self-attention can stably represent nodes.

[0103] For the output features of the middle layer, each graph convolution layer uses K self-attentions to calculate and concatenate the results of the attention heads to obtain the output vector. The output vectors of each attention head are averaged to obtain the output results of each graph convolution layer. In one embodiment, K represents the number of attentions in each graph convolution layer, which can also be understood as K-head attention, and can be limited according to specific circumstances.

[0104] In S413, the multiple attention mechanism graph convolution layers can be three graph convolution layers. The number of graph convolution layers is not limited and can be limited according to actual conditions. The input point feature vector passes through the three graph convolution layers, outputting node hidden layer features, and then obtaining node hidden layer features output by the multi-layer graph convolution layer. Maximum pooling and average pooling are performed respectively to obtain two pooled output features. Both maximum pooling and average pooling are performed in parallel on the node hidden layer.

[0105] The graph convolution layer consists of three layers of graph convolution layers. Each layer of graph convolution layer uses a multi-head attention mechanism. The input H of the first layer of graph convolution layer is (0) =[h0 (0) ,h1 (0) ,...,h i (0) ,...,h t (0) ], output H (1) =[h0 (1) ,h1 (1) ,...,h i (1) ,...,h t (1) ],

[0106] The input H of the second graph convolutional layer (1) =[h0 (1) ,h1 (1) ,...,h i (1) ,...,h t (1) ], output H (2) =[h0 (2) ,h1 (2) ,...,h i (2) ,...,h t (2) ],

[0107] The input H of the third graph convolutional layer (2) =[h0 (2) ,h1(2) ,...,h i (2) ,...,h t (2) ], output H (3) =[h0 (3) ,h1 (3) ,...,h i (3) ,...,h t (3) ],

[0108] H (3) =[h0 (3) ,h1 (3) ,...,h i (3) ,...,h t (3) ] is the node hidden layer feature output by the 3-layer graph convolution layer. (3) =[h0 (3) ,h1 (3) ,...,h i (3) ,...,h t (3) ] Perform maximum pooling and average pooling respectively to obtain two pooling output features.

[0109] By splicing the two pooling output features after the two pooling processes, inputting them into the softmax recognition function, the probability of the vehicle appearing at each camera point at the next moment is output, and then the final vehicle prediction result can be output. For example: the output probability of the vehicle appearing at each camera point at the next moment is: camera point v 56 , the probability is 96%; the camera point v 73 , the probability is 3%; the camera point v 431 , with a probability of 1%. The final output vehicle prediction result is the camera point v 56 .

[0110] Starting from the entire monitoring system, based on the relationship between all camera points, the camera points of the entire monitoring network are covered. And through specific data to characterize the information of the camera points of the monitoring network and the relationship between each camera point, the relationship between all camera points is fully considered to perform model prediction. Based on the analysis of the historical records of vehicles appearing at each camera point, the training and analysis of large-scale data using deep learning is made, so that the point prediction model has a strong learning ability, a wide coverage range, and good adaptability, and can more accurately predict the camera point where the vehicle may appear at the next moment. Therefore, the vehicle prediction method provided by this application can improve the accuracy of vehicle prediction, can be applied to large-scale vehicle monitoring systems, and can achieve accurate search of target vehicles. Therefore, the vehicle prediction method provided by this application avoids the use of manual analysis methods, will not miss the camera points where the vehicle may appear, and can be used in practical application environments such as vehicle query and parking anomaly detection.

[0111] In one embodiment, the point feature vectors corresponding to the n camera points in the camera point map obtained by the word skipping model can be stored in a dictionary structure, such as {v j :x j}, v j Indicates the camera position, x j Indicates v j The corresponding point feature vector. According to the word skipping model, the point feature vector corresponding to each camera point in the camera point map can be obtained. When the point prediction model is used to predict the point, the v j Find the corresponding x j , and conduct subsequent point prediction processing.

[0112] In one embodiment, S420, based on the sequence of multiple point feature vectors and the camera point prediction results, the attention mechanism graph convolutional network is trained to obtain the parameters of the attention mechanism graph convolutional network to obtain a trained point prediction model, including:

[0113] S421: Based on the probability of the vehicle appearing at each camera point at the next moment, a target loss function of the point prediction model is constructed. The parameters of each layer of the multi-layer graph convolution layer are adjusted according to the value of the target loss function of the point prediction model to obtain a trained point prediction model. The target loss function of the point prediction model is:

[0114]

[0115] Among them, M represents the number of point feature vectors in the training set formed by multiple point feature vectors, N represents the number of camera points in the camera point map, and y ijRepresents the label of the point feature vector i in the training set. If the true prediction result of the point feature vector i is the camera point j, the value is 1, otherwise it is 0. ij It represents the probability that the point feature vector i corresponding to the vehicle appears at the camera point j at the next moment.

[0116] The cross-entropy loss function can be used as the objective loss function for the point prediction model. During the training process, the objective loss function will continuously decrease and eventually reach a state of equilibrium. When the objective loss function reaches equilibrium, the point prediction model is optimized, and the trained point prediction model is obtained, which can make the prediction model more stable and reliable.

[0117] In one embodiment, S50 , the camera position where the vehicle will appear at the next moment is predicted based on the position prediction model.

[0118] After the vectorization model and point prediction model are trained, the historical camera point sequence (also understood as the original sequence or camera point sequence) of a vehicle appearing in the monitoring network is obtained. Using the skip-gram model, the point feature vector corresponding to each camera point in the camera point sequence can be obtained. A camera point sequence is formed based on the historical camera point sequence, and the point feature vector corresponding to each camera point can be retrieved. Multiple point feature vectors form an input sequence and are input into the point prediction model. Based on the point prediction model, the probability of the vehicle appearing at each point at the next moment is obtained, and the camera point at which the vehicle will appear at the next moment is further determined.

[0119] See Figure 7 In one embodiment, the present application provides a vehicle prediction device 100. The vehicle prediction device 100 includes a camera point map generation module 10, a camera point sequence acquisition module 20, a vectorized model generation module 30, a point prediction model generation module 40, and a prediction module 50.

[0120] The camera point map generation module 10 is used to obtain multiple camera points of the monitoring system and the node relationships between the multiple camera points, and to construct a camera point map based on the multiple camera points and the node relationships. The camera point sequence acquisition module 20 is used to obtain the historical records of vehicles being photographed in the monitoring system based on the camera point map to form multiple camera point sequences. The vectorization model generation module 30 is used to construct a vectorization model training set based on the multiple camera point sequences, train to form a vectorization model based on the vectorization model training set, and perform feature vectorization on each camera point in the camera point map based on the vectorization model to obtain multiple point feature vectors. The point prediction model generation module 40 is used to construct a point prediction model training set based on the multiple point feature vectors, and train to form a point prediction model based on the point prediction model training set. The prediction module 50 is used to predict the camera point where the vehicle will appear at the next moment based on the vectorization model and the point prediction model.

[0121] In this embodiment, the description of the camera point map generation module 10 can refer to the description of S10 in the above embodiment. The description of the camera point sequence acquisition module 20 can refer to the description of S20 in the above embodiment. The description of the vectorized model generation module 30 can refer to the description of S30 in the above embodiment. The description of the point prediction model generation module 40 can refer to the description of S50 in the above embodiment. The description of the prediction module 50 can refer to the description of S60 in the above embodiment.

[0122] In one embodiment, the vectorized model generation module 30 includes a parameter setting module (not labeled in the figure), a sample generation module (not labeled in the figure), and a model training module (not labeled in the figure). The parameter setting module is used to set the window size for training the word-skipping model. The sample generation module is used to extract samples from each of the camera point sequences according to the window size to form multiple groups of model training samples. The model training module is used to train the word-skipping model based on the multiple groups of model training samples.

[0123] In this embodiment, the description of the parameter setting module can refer to the description of S310 in the above embodiment. The description of the sample generation module can refer to the description of S320 in the above embodiment. The description of the model training module can refer to the description of S330 in the above embodiment.

[0124] In one embodiment, the model training module (not labeled in the figure) includes a model construction module (not labeled in the figure) and a first loss function adjustment module (not labeled in the figure). The model construction module is used to input the single camera point into the word-skipping model and output the predicted context point sequence; wherein, the word-skipping model is a three-layer neural network. The first loss function module is used to train the word-skipping model based on the single camera point and the predicted context point sequence, and construct the target loss function of the word-skipping model, and iteratively adjust the parameters of each layer of the neural network of the word-skipping model through the gradient descent algorithm during training. Wherein, the target loss function of the word-skipping model is:

[0125]

[0126]

[0127] v in represents a single camera point input to the word skipping model, (v out,1 ,v out,2 ,...,v out,t ) represents the v output by the skip-word model in The predicted context point sequence corresponding to a single camera point, t represents the window size, h represents the vector of dimension d of the hidden layer, W out represents the d×n dimensional connection weights of the hidden layer.

[0128] In this embodiment, the description of the hidden layer parameter acquisition module can refer to the description of S331 in the above embodiment. The description of the first loss function adjustment module can refer to the description of S332 in the above embodiment.

[0129] In one embodiment, the point prediction model generation module 40 includes a point prediction model construction module and a point prediction model training module. The point prediction model construction module (not shown in the figure) is used to input multiple point feature vectors into the attention mechanism graph convolutional network, output multiple node hidden layer features, and input multiple node hidden layer features into the classification layer to output the camera point prediction results. The point prediction model training module (not shown in the figure) is used to train the attention mechanism graph convolutional network based on the multiple point feature vectors and the camera point prediction results, obtain the parameters of the attention mechanism graph convolutional network, and obtain a trained point prediction model.

[0130] In one embodiment, the point prediction model construction module includes an attention coefficient construction module, a multi-head attention mechanism graph convolution layer feature acquisition module, a multi-layer graph convolution layer construction module, a two-pooling output feature acquisition module, a probability acquisition module, and a camera point output module. The attention coefficient construction module is used to construct an attention coefficient based on a graph convolution network, a feedforward neural network, and a leaky rectified linear unit function; the attention coefficient is:

[0131]

[0132] in, represents a feedforward neural network, W represents a shared weight matrix, Represents the node hidden layer feature corresponding to node i output by the l-th layer graph convolution, represents the node hidden layer feature corresponding to node j output by the l-th layer graph convolution, the || symbol represents vector concatenation, and the node i and the node j are adjacent nodes;

[0133] The multi-head attention mechanism graph convolution layer feature acquisition module is used to obtain the node hidden layer feature corresponding to the node i output by the l+1th layer graph convolution layer of the multi-head attention mechanism according to the attention coefficient; the node hidden layer feature corresponding to the node i output by the l+1th layer graph convolution layer of the multi-head attention mechanism is:

[0134]

[0135]

[0136] Among them, σ(·) represents the activation function, N i represents the set of nodes in the neighborhood of the node i, and K represents the number of multi-head attention.

[0137] The multi-layer graph convolution layer construction module is used to construct a multi-layer graph convolution layer according to the node hidden layer features corresponding to the node i output by the l+1th layer graph convolution layer of the multi-head attention mechanism;

[0138] The two-pooled output feature acquisition module performs maximum pooling and average pooling on the node hidden layer features output by the multi-layer graph convolution layer to obtain two-pooled output features. The probability acquisition module inputs the two-pooled output features into the recognition function and outputs the probability of the vehicle appearing at each camera point at the next moment. The camera point output module obtains the camera point where the vehicle will appear at the next moment based on the probability of the vehicle appearing at each camera point at the next moment.

[0139] In this embodiment, the description of the attention coefficient construction module can refer to the description of S411 in the above embodiment. The description of the multi-head attention mechanism graph convolution layer feature acquisition module can refer to the description of S412 in the above embodiment. The description of the multi-layer graph convolution layer construction module can refer to the description of S413 in the above embodiment. The description of the two-pooling output feature acquisition module can refer to the description of S414 in the above embodiment. The description of the probability acquisition module can refer to the description of S415 in the above embodiment. The description of the camera point output module can refer to the description of S416 in the above embodiment.

[0140] In one embodiment, the point prediction model training module includes a second loss function module (not marked in the figure). The second loss function module is used to construct a target loss function of the point prediction model based on the probability of the vehicle appearing at each camera point at the next moment, and adjust the parameters of each layer of the multi-layer graph convolution layer according to the value of the target loss function of the point prediction model to obtain a trained point prediction model; the target loss function of the point prediction model is:

[0141]

[0142] Wherein, M represents the number of point feature vectors in the training set formed by the plurality of point feature vectors, N represents the number of camera points in the camera point map, and y ij represents the label of the point feature vector i in the training set. If the true prediction result of the point feature vector i is the camera point j, the value is 1, otherwise it is 0. ij Represents the probability that the point feature vector i corresponding to the vehicle appears at the camera point j at the next moment.

[0143] In this embodiment, the relevant description of the second loss function adjustment module can refer to the relevant description of S421 in the above embodiment.

[0144] In one embodiment, the camera point map generation module 10 includes a camera point set generation module (not labeled in the figure), a node edge set generation module (not labeled in the figure), and a construction module (not labeled in the figure).

[0145] The camera point set generation module is used to obtain the multiple camera points to form a camera point set V, with each camera point set being considered as a node. The node edge set generation module is used to obtain the order in which multiple vehicles are captured between multiple nodes, form node relationships between adjacent nodes, and form a node edge set E based on these node relationships. The construction module is used to construct the camera point graph G(V,E) based on the camera point set V and the node edge set E.

[0146] In this embodiment, the description of the camera point set generation module can refer to the description of S110 in the above embodiment. The description of the node edge set generation module can refer to the description of S120 in the above embodiment. The description of the construction module can refer to the description of S130 in the above embodiment.

[0147] In one embodiment, the camera point sequence acquisition module 20 includes a record acquisition module (not labeled in the figure). The record acquisition module is configured to acquire a camera point sequence corresponding to the vehicle based on the camera point map G(V, E). The camera point sequence includes records of the same vehicle captured at multiple camera points within a time period.

[0148] In this embodiment, the relevant description of the record acquisition module may refer to the relevant description of S210 in the above embodiment.

[0149] In the various embodiments described above, the specific order or hierarchy of steps in the disclosed processes is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process can be rearranged without departing from the scope of protection of this disclosure. The accompanying method claims present the elements of the various steps in an exemplary order and are not intended to be limited to the specific order or hierarchy described.

[0150] Those skilled in the art will also appreciate that the various illustrative logical blocks, units, and steps listed in the embodiments of the present application can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly demonstrate the interchangeability of hardware and software, the various illustrative components, units, and steps described above have generally described their functions. Whether such functions are implemented by hardware or software depends on the specific application and the design requirements of the entire system. Those skilled in the art may use various methods to implement the described functions for each specific application, but such implementation should not be understood as exceeding the scope of protection of the embodiments of the present application.

[0151] The various illustrative logic blocks described in the embodiments of the present application, or units can be implemented or operated by the design of a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic, a discrete hardware component, or any combination thereof. The general-purpose processor can be a microprocessor, alternatively, the general-purpose processor can also be any traditional processor, controller, microcontroller or state machine. The processor can also be implemented by a combination of computing devices, such as a digital signal processor and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a digital signal processor core, or any other similar configuration to implement.

[0152] The steps of the methods or algorithms described in the embodiments of the present application can be directly embedded in hardware, a software module executed by a processor, or a combination of the two. The software module can be stored in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. Exemplarily, the storage medium can be connected to the processor so that the processor can read information from the storage medium and write information to the storage medium. Alternatively, the storage medium can also be integrated into the processor. The processor and the storage medium can be provided in an ASIC, which can be provided in a user terminal. Alternatively, the processor and the storage medium can also be provided in different components in the user terminal.

[0153] The specific implementation methods described above further illustrate the purpose, technical solutions and beneficial effects of this application. It should be understood that the above description is only the specific implementation methods of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this application should be included in the scope of protection of this application.

Claims

1. A vehicle prediction method, characterized in that: include: Acquire multiple camera points of the monitoring system and node relationships between the multiple camera points, and construct a camera point map based on the multiple camera points and the node relationships; Based on the camera point map, a historical record of vehicles being photographed in the monitoring system is obtained to form multiple camera point sequences; Constructing a vectorized model training set based on the multiple camera point sequences, training a vectorized model based on the vectorized model training set, and performing feature vectorization on each camera point in the camera point map based on the vectorized model to obtain multiple point feature vectors; Constructing a point prediction model training set based on the plurality of point feature vectors, and training a point prediction model based on the point prediction model training set; The camera position where the vehicle will appear at the next moment is predicted based on the position prediction model.

2. The vehicle prediction method according to claim 1, characterized in that: The vectorized model is a word-skipping model, and constructing a vectorized model training set according to the plurality of camera point sequences, and training to form a vectorized model according to the vectorized model training set includes: Setting a window size for training the word skipping model; Extracting samples from each of the camera point sequences according to the window size to form multiple groups of model training samples; wherein each group of model training samples includes a single camera point and a corresponding predicted context point sequence; The skip-word model is trained and formed according to the multiple groups of model training samples.

3. The vehicle prediction method according to claim 2, characterized in that: The step of training the skip-word model according to the plurality of groups of model training samples comprises: Inputting the single camera point into the word-skipping model and outputting the predicted context point sequence; wherein the word-skipping model is a three-layer neural network; The word-skipping model is trained based on the single camera point and the predicted context point sequence, and a target loss function of the word-skipping model is constructed. During training, the parameters of each layer of the neural network of the word-skipping model are iteratively adjusted using a gradient descent algorithm. The target loss function of the word-skipping model is: v in represents a single camera point input to the word skipping model, (v out,1 ,v out,2 ,...,v out,t ) represents the v output by the skip-word model in The predicted context point sequence corresponding to a single camera point, t represents the window size, h represents the vector of dimension d of the hidden layer, W out represents the d×n dimensional connection weights of the hidden layer.

4. The vehicle prediction method according to claim 1, characterized in that: The step of constructing a point prediction model training set based on the plurality of point feature vectors, and training a point prediction model based on the point prediction model training set includes: Input the multiple point feature vectors into the attention mechanism graph convolutional network, output multiple node hidden layer features, and input the multiple node hidden layer features into the classification layer, outputting the camera point prediction results; According to the multiple point feature vectors and the camera point prediction results, the attention mechanism graph convolutional network is trained, the parameters of the attention mechanism graph convolutional network are obtained, and a trained point prediction model is obtained.

5. The vehicle prediction method according to claim 4, characterized in that: The step of inputting the plurality of point feature vectors into the attention mechanism graph convolutional network, outputting a plurality of node hidden layer features, and inputting the plurality of node hidden layer features into the classification layer, and outputting the camera point prediction results, includes: According to the graph convolutional network, feedforward neural network and leaky rectified linear unit function, the attention coefficient is constructed; the attention coefficient is: in, represents a feedforward neural network, W represents a shared weight matrix, Represents the node hidden layer features corresponding to node i output by the l-th layer graph convolution, represents the node hidden layer feature corresponding to node j output by the l-th layer graph convolution, the || symbol represents vector concatenation, and the node i and the node j are adjacent nodes; According to the attention coefficient, the node hidden layer feature corresponding to the node i output by the l+1th graph convolution layer of the multi-head attention mechanism is obtained; the node hidden layer feature corresponding to the node i output by the l+1th graph convolution layer of the multi-head attention mechanism is: Among them, σ(·) represents the activation function, N i represents the set of nodes in the neighborhood of the node i, and K represents the number of multi-head attentions; Construct a multi-layer graph convolution layer based on the node hidden layer features corresponding to node i output by the l+1th graph convolution layer of the multi-head attention mechanism; Performing maximum pooling and average pooling on the node hidden layer features output by the multi-layer graph convolution layer to obtain two-pooled output features; Input the two pooled output features into the recognition function, and output the probability of the vehicle appearing at each camera point at the next moment; The camera point where the vehicle appears at the next moment is obtained according to the probability that the vehicle appears at each camera point at the next moment.

6. The vehicle prediction method according to claim 5, characterized in that: The method further includes training the attention mechanism graph convolutional network according to the plurality of point feature vectors and the camera point prediction results, obtaining parameters of the attention mechanism graph convolutional network, and obtaining a trained point prediction model. According to the probability of the vehicle appearing at each camera point at the next moment, the target loss function of the point prediction model is constructed, and the parameters of each layer of the multi-layer graph convolution layer are adjusted according to the value of the target loss function of the point prediction model to obtain the trained point prediction model; the target loss function of the point prediction model is: Wherein, M represents the number of point feature vectors in the training set formed by the plurality of point feature vectors, N represents the number of camera points in the camera point map, and y ij represents the label of the point feature vector i in the training set. If the true prediction result of the point feature vector i is the camera point j, the value is 1, otherwise it is 0. ij Represents the probability that the point feature vector i corresponding to the vehicle appears at the camera point j at the next moment.

7. The vehicle prediction method according to claim 1, characterized in that: The acquiring of a plurality of camera positions of the monitoring system and node relationships between the plurality of camera positions, and constructing a camera position map according to the plurality of camera positions and the node relationships, includes: Acquire the plurality of camera points to form a camera point set V, and treat each of the camera points as a node; Obtaining the order in which multiple vehicles are captured between multiple nodes, forming a node relationship between each adjacent node, and forming a node edge set E based on the node relationship; The camera point graph G(V,E) is constructed according to the camera point set V and the node edge set E.

8. The vehicle prediction method according to claim 7, characterized in that: The step of obtaining a plurality of camera point sequences according to the camera point map includes: According to the camera point map G(V,E), obtain the camera point sequence corresponding to the vehicle; The camera point sequence includes records of the same vehicle being captured between multiple camera points within a time period.

9. A vehicle prediction device, characterized in that: include: A camera point map generation module is used to obtain a plurality of camera points of the monitoring system and node relationships between the plurality of camera points, and to construct a camera point map based on the plurality of camera points and the node relationships; A camera point sequence acquisition module is used to acquire the historical records of vehicles being photographed in the monitoring system based on the camera point map to form multiple camera point sequences; a vectorization model generation module, configured to construct a vectorization model training set based on the plurality of camera point sequences, train a vectorization model based on the vectorization model training set, and perform feature vectorization on each camera point in the camera point map based on the vectorization model to obtain a plurality of point feature vectors; A point prediction model generation module is used to construct a point prediction model training set based on the multiple point feature vectors, and train a point prediction model based on the point prediction model training set; The prediction module is used to predict the camera point where the vehicle will appear at the next moment based on the point prediction model.

10. The vehicle prediction device according to claim 9, characterized in that The vectorized model generation module includes: A parameter setting module, used to set the window size for training the word skipping model; A sample generation module is used to extract samples from each of the camera point sequences according to the window size to form multiple groups of model training samples; wherein each group of model training samples includes a single camera point and a corresponding predicted context point sequence; The model training module is used to train and form the skip-word model based on the multiple groups of model training samples.

11. The vehicle prediction device according to claim 10, characterized in that: The model training module includes: A model building module, configured to input the single camera point into the word-skipping model and output the predicted context point sequence; wherein the word-skipping model is a three-layer neural network; a first loss function module, configured to train the skip-gram model based on the single camera point and the predicted context point sequence, construct a target loss function for the skip-gram model, and iteratively adjust the neural network parameters of each layer of the skip-gram model using a gradient descent algorithm during training; Among them, the target loss function of the word skipping model is: v in represents a single camera point input to the word skipping model, (v out,1 ,v out,2 ,...,v out,t ) represents the v output by the skip-word model in The predicted context point sequence corresponding to a single camera point, t represents the window size, h represents the vector of dimension d of the hidden layer, W out represents the d×n dimensional connection weights of the hidden layer.

12. The vehicle prediction device according to claim 9, characterized in that The point prediction model generation module includes: A point prediction model construction module is used to input the multiple point feature vectors into the attention mechanism graph convolutional network, output multiple node hidden layer features, and input the multiple node hidden layer features into the classification layer to output the camera point prediction result; The point prediction model training module is used to train the attention mechanism graph convolutional network based on the multiple point feature vectors and the camera point prediction results, obtain the parameters of the attention mechanism graph convolutional network, and obtain a trained point prediction model.

13. The vehicle prediction device according to claim 12, characterized in that: The point prediction model construction module includes: The attention coefficient construction module is used to construct the attention coefficient based on the graph convolutional network, the feedforward neural network and the leaky rectified linear unit function; the attention coefficient is: in, represents a feedforward neural network, W represents a shared weight matrix, Represents the node hidden layer features corresponding to node i output by the l-th layer graph convolution, represents the node hidden layer feature corresponding to node j output by the l-th layer graph convolution, the || symbol represents vector concatenation, and the node i and the node j are adjacent nodes; The multi-head attention mechanism graph convolution layer feature acquisition module is used to obtain the node hidden layer feature corresponding to the node i output by the l+1th layer graph convolution layer of the multi-head attention mechanism according to the attention coefficient; the node hidden layer feature corresponding to the node i output by the l+1th layer graph convolution layer of the multi-head attention mechanism is: Among them, σ(·) represents the activation function, N i represents the set of nodes in the neighborhood of the node i, and K represents the number of multi-head attentions; A multi-layer graph convolutional layer construction module is used to construct a multi-layer graph convolutional layer based on the node hidden layer features corresponding to node i output by the l+1th layer graph convolutional layer of the multi-head attention mechanism; A two-pooling output feature acquisition module is used to perform maximum pooling and average pooling on the node hidden layer features output by the multi-layer graph convolution layer to obtain two-pooling output features; A probability acquisition module, configured to input the two pooled output features into a recognition function and output the probability of the vehicle appearing at each camera point at the next moment; The camera point output module is used to obtain the camera point where the vehicle appears at the next moment according to the probability that the vehicle appears at each camera point at the next moment.

14. The vehicle prediction device according to claim 13, wherein: The point prediction model training module includes: The second loss function module is used to construct the target loss function of the point prediction model according to the probability of the vehicle appearing at each camera point at the next moment, and adjust the parameters of each layer of the multi-layer graph convolution layer according to the value of the target loss function of the point prediction model to obtain the trained point prediction model; the target loss function of the point prediction model is: Wherein, M represents the number of point feature vectors in the training set formed by the plurality of point feature vectors, N represents the number of camera points in the camera point map, and y ij represents the label of the point feature vector i in the training set. If the true prediction result of the point feature vector i is the camera point j, the value is 1, otherwise it is 0. ij Represents the probability that the point feature vector i corresponding to the vehicle appears at the camera point j at the next moment.

15. The vehicle prediction device according to claim 9, characterized in that The camera point map generation module includes: A camera point set generation module is used to obtain the multiple camera points to form a camera point set V, and each of the camera points is regarded as a node; A node edge set generation module is used to obtain the order in which multiple vehicles are captured between multiple nodes, form a node relationship between each adjacent node, and form a node edge set E based on the node relationship; A construction module is used to construct the camera point graph G(V, E) according to the camera point set V and the node edge set E.

16. The vehicle prediction device according to claim 15, characterized in that The camera point sequence acquisition module includes: The record acquisition module is used to obtain a camera point sequence corresponding to the vehicle according to the camera point map G(V, E), where the camera point sequence includes records of the same vehicle being captured between multiple camera points within a time period.

Images