Vehicle parking position prediction method based on multiple modes and related equipment

Abstract

The invention relates to the technical field of traffic big data, and provides a vehicle parking position prediction method based on multiple modes and related equipment. The method comprises the following steps: acquiring a plurality of parking points of a target vehicle and parking point information of each parking point based on all track points of the target vehicle; constructing a heterogeneous information network; mapping the heterogeneous information network to obtain an embedded vector of the node; splicing the embedded vector corresponding to the parking point of the target vehicle and the embedded vector corresponding to the identification information of the target vehicle to obtain a parking point vector, and combining all the parking point vectors into a vector trajectory; based on the weather information, the stay point information and the vector trajectory, using the prediction model to predict a stay position to obtain a predicted stay position; and training a prediction model by using the predicted parking position, and predicting the to-be-predicted vehicle by using the trained prediction model to obtain a final predicted parking position. According to the method, the accuracy of vehicle parking position prediction can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of traffic big data, and in particular to a vehicle stop position prediction method based on multimodality and related equipment. Background Art

[0002] Accurately predicting the future location of a vehicle plays a vital role in many fields such as service recommendation, urban planning, and traffic management. First, in the field of service recommendation, accurate prediction of the vehicle's location can provide drivers with a more personalized and accurate service experience, such as surrounding facility recommendations and parking services, thereby improving user satisfaction and loyalty. Second, in terms of urban planning, the prediction of vehicle location helps the government and relevant departments better understand the distribution of urban traffic flow, optimize the traffic network and traffic system, rationally plan the layout of public service facilities, and improve the livability and sustainability of the city. Finally, in the field of traffic management, predicting the location of a vehicle can assist in traffic planning. Through the analysis of real-time traffic data, it can optimize traffic flow, reduce congestion, improve road use efficiency, and provide technical support for traffic diversion and environmental governance. In addition, the mining of vehicle trajectory data and the identification of travel patterns are of great significance for building smart cities and realizing smart transportation, and can provide data-driven insights for traffic management and decision-making.

[0003] In recent years, predicting the future location of a vehicle based on its historical trajectory data has become a research hotspot. Existing studies have predicted the user's future location by mining the knowledge contained in the vehicle's historical trajectory data. However, few scholars have conducted in-depth research on how to use multimodal data to predict vehicle location at the same time. Multimodal learning can enhance the model's ability to understand vehicle behavior by fusing data from different modalities. For example, combining the vehicle's historical trajectory with surrounding environmental information can more accurately predict the vehicle's intention to stay. However, the integration of multimodal data faces many challenges, including data synchronization, semantic alignment between modalities, and dynamic fusion of information. Therefore, it is difficult to predict the vehicle's stop location based on multimodal data, resulting in low accuracy in vehicle stop location prediction. Summary of the invention

[0004] The present application provides a vehicle stop position prediction method and related equipment based on multi-modality, which can solve the problem of low accuracy of vehicle stop position prediction.

[0005] In a first aspect, an embodiment of the present application provides a vehicle stop position prediction method based on multimodality, the vehicle stop position prediction method comprising:

[0006] Obtain weather information of the target area at multiple historical moments, and the track points of each target vehicle in the target area at all historical moments, and based on all the track points of each target vehicle, obtain multiple stop points of each target vehicle and the stop point information of each stop point; the stop point information includes the stop time, stop time, location and corresponding interest point attributes of the stop point;

[0007] A heterogeneous information network is constructed according to all target vehicles, all stop points, all stop point information, and all interest point attributes; the attribute of each node in the heterogeneous information network is one of the identification information of multiple target vehicles, multiple stop points, and multiple interest point attributes, and the identification information of all target vehicles, all stop points, and all interest point attributes have a corresponding node, and the edge between the nodes is the association relationship between the attributes of the corresponding two nodes;

[0008] Map the heterogeneous information network to obtain the embedding vector of each node in the heterogeneous information network;

[0009] For each target vehicle, the embedded vector corresponding to each stop point of the target vehicle is concatenated with the embedded vector corresponding to the identification information of the target vehicle to obtain multiple stop point vectors, and all the stop point vectors are combined into a vector trajectory;

[0010] Based on all weather information, all stop point information, and all vector trajectories, a prediction model is used to predict the stop position of each target vehicle to obtain the predicted stop position of each target vehicle;

[0011] The prediction model is trained using all predicted stop positions to obtain a trained prediction model, and the trained prediction model is used to predict the stop position of the vehicle to be predicted to obtain the final predicted stop position of the vehicle to be predicted.

[0012] Optionally, based on all the track points of each target vehicle, multiple stop points of each target vehicle and the stop point information of each stop point are obtained, including:

[0013] For each target vehicle, perform the following steps:

[0014] Calculate the distance difference between every two trajectory points of the target vehicle;

[0015] For each distance difference, if the distance difference is less than the distance difference threshold, a stop point is generated, and the average longitude and latitude between the two trajectory points corresponding to the distance difference is used as the location of the stop point, the time difference between the historical moments corresponding to the two trajectory points is used as the stop duration of the stop point, and the earliest historical moment between the two historical moments corresponding to the two trajectory points is used as the stop time of the stop point, and the attributes of the point of interest closest to the stop point are obtained;

[0016] The stay duration, stay time, location and corresponding point of interest attributes of each stay point are integrated to obtain the stay point information of the stay point.

[0017] Optionally, the distance difference between every two trajectory points of the target vehicle is calculated, including:

[0018] By formula:

[0019]

[0020] Calculate the distance difference d between the nth trajectory point and the mth trajectory point nm ;

[0021] Where r is the radius of the earth, represents the latitude of the mth trajectory point, represents the latitude of the nth trajectory point, λ m represents the longitude of the mth trajectory point, λ n Represents the longitude of the nth trajectory point, n,m∈N,n≠m, N represents the set of trajectory point numbers of the target vehicle.

[0022] Optionally, the heterogeneous information network is mapped to obtain an embedding vector of each node in the heterogeneous information network, including:

[0023] By formula:

[0024]

[0025] Calculate the embedding vector value of the i-th node at the x+1 layer

[0026] in, represents the embedding vector value of the i-th node in the x-th layer, and Both represent learnable parameters, represents the mapping embedding vector of the i-th node mapped to the j-th node in the x+1-th layer, represents the parameter training matrix, N i represents the set of all adjacent nodes of the ith node in the heterogeneous information network, v j represents the jth node in the heterogeneous information network, v i represents the i-th node in a heterogeneous information network, represents the correlation coefficient between the i-th node and the j-th node, θ represents the trainable parameter, σ represents the activation function, i, j∈M, i≠j, M represents the set of node numbers in the heterogeneous information network, || represents the connection operation, x=0,1,...,X-1, X represents the last layer of the mapping, when x=0, are all initial vectors. When x=X-1, the embedding vector value of the i-th node in the X-th layer is as the embedding vector of the i-th node.

[0027] Optionally, combine all stop point vectors into a vector trajectory, including:

[0028] All the stop point vectors are sorted according to the order of the stop times corresponding to them from early to late to obtain the vector trajectory.

[0029] Optionally, based on all weather information, all stop point information, and all vector trajectories, a prediction model is used to predict the stop position of each target vehicle to obtain a predicted stop position of each target vehicle, including:

[0030] For each stop point, embed the stop time and stop duration in the stop point information to obtain a time embedding vector, and embed the weather information corresponding to the stop time of the stop point to obtain a weather embedding vector;

[0031] For each vector trajectory, concatenate each stop point vector in the vector trajectory with the time embedding vector and weather embedding vector corresponding to the stop point vector to obtain the overall vector trajectory;

[0032] The prediction model is used to calculate each overall vector trajectory to obtain the final vector trajectory corresponding to each target vehicle, and each final vector trajectory is predicted to obtain the predicted stop position of each target vehicle.

[0033] Optionally, each final vector trajectory is predicted to obtain the predicted stop position of each target vehicle, including:

[0034] By formula:

[0035] l = tanh(W o ×h+b o )

[0036] Calculate the predicted stop position l of the target vehicle;

[0037] Where h represents the final vector trajectory of the target vehicle, W o and b o All are parameters.

[0038] In a second aspect, an embodiment of the present application provides a vehicle stop position prediction device based on multi-modality, comprising:

[0039] An acquisition module is used to acquire weather information of a target area at multiple historical moments, and track points of each target vehicle in the target area at all historical moments, and based on all track points of each target vehicle, multiple stop points of each target vehicle and stop point information of each stop point are acquired; the stop point information includes the stop time, stop time, location and corresponding interest point attributes of the stop point;

[0040] A construction module is provided to construct a heterogeneous information network according to all target vehicles, all stop points, all stop point information, and all interest point attributes; the attribute of each node in the heterogeneous information network is one of the identification information of multiple target vehicles, multiple stop points, and multiple interest point attributes, and the identification information of all target vehicles, all stop points, and all interest point attributes have a corresponding node, and the edge between the nodes is the association relationship between the attributes of the corresponding two nodes;

[0041] A mapping module maps the heterogeneous information network to obtain the embedding vector of each node in the heterogeneous information network;

[0042] A concatenation module concatenates, for each target vehicle, the embedded vector corresponding to each stop point of the target vehicle with the embedded vector corresponding to the identification information of the target vehicle to obtain multiple stop point vectors, and combines all the stop point vectors into a vector trajectory;

[0043] The stop position prediction module predicts the stop position of each target vehicle using a prediction model based on all weather information, all stop point information, and all vector trajectories to obtain the predicted stop position of each target vehicle;

[0044] The training module uses all predicted stop positions to train the prediction model to obtain a trained prediction model, and uses the trained prediction model to predict the stop position of the vehicle to be predicted to obtain the final predicted stop position of the vehicle to be predicted.

[0045] In a third aspect, an embodiment of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned multi-modal-based vehicle stop position prediction method when executing the above-mentioned computer program.

[0046] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, it implements the above-mentioned multi-modal based vehicle stop position prediction method.

[0047] The above solution of the present application has the following beneficial effects:

[0048] In an embodiment of the present application, multiple stop points of each target vehicle and the stop point information of each stop point are obtained based on all trajectory points of each target vehicle, and then a heterogeneous information network is constructed according to all target vehicles, all stop points, all stop point information, and all interest point attributes, and then the heterogeneous information network is mapped to obtain an embedding vector of each node in the heterogeneous information network, and then for each target vehicle, the embedding vector corresponding to each stop point of the target vehicle is concatenated with the embedding vector corresponding to the identification information of the target vehicle to obtain multiple stop point vectors, and all stop point vectors are combined into a vector trajectory, and then based on all weather information, all stop point information, and all vector trajectories, the stop position of each target vehicle is predicted using a prediction model to obtain a predicted stop position of each target vehicle, and finally all predicted stop positions are used to train the prediction model to obtain a trained prediction model, and the trained prediction model is used to predict the stop position of the vehicle to be predicted to obtain the final predicted stop position of the vehicle to be predicted. Among them, the constructed heterogeneous information network includes three aspects of information: identification information, stop points, and points of interest attributes of the target vehicle, which improves the multimodal information richness of the heterogeneous information network. The embedding vector of the node obtained based on the information-rich heterogeneous information network mines the correlation between the attributes of the target vehicle, stop points, and points of interest, improves the accuracy and comprehensiveness of the embedding vector, and predicts the stop location based on accurate embedding vectors, weather information, and stop point information. The information of embedded vectors, weather information, and stop point information is integrated to improve the information richness and accuracy of the data during prediction, thereby improving the accuracy of vehicle stop location prediction.

[0049] Other beneficial effects of the present application will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0051] Figure 1 A flowchart of a vehicle stop position prediction method based on multi-modality provided in one embodiment of the present application;

[0052] Figure 2 A schematic diagram of a specific process of a vehicle stop position prediction method based on multi-modality provided in an embodiment of the present application;

[0053] Figure 3 A schematic diagram of the structure of a vehicle stop position prediction device based on multi-modality provided in one embodiment of the present application;

[0054] Figure 4 A schematic diagram of the structure of a terminal device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0055] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0056] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0057] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0058] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0059] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0060] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0061] In order to solve the problem of low accuracy of existing vehicle stop position prediction, an embodiment of the present application provides a vehicle stop position prediction method based on multimodality. The vehicle stop position prediction method obtains multiple stop points of each target vehicle and the stop point information of each stop point based on all trajectory points of each target vehicle, then constructs a heterogeneous information network according to all target vehicles, all stop points, all stop point information, and all interest point attributes, and then maps the heterogeneous information network to obtain the embedding vector of each node in the heterogeneous information network, and then respectively for each target vehicle, the embedding vector corresponding to each stop point of the target vehicle is spliced ​​with the embedding vector corresponding to the identification information of the target vehicle to obtain multiple stop point vectors, and all stop point vectors are combined into vector trajectories, and then based on all weather information, all stop point information, and all vector trajectories, the stop position of each target vehicle is predicted by using a prediction model to obtain the predicted stop position of each target vehicle, and finally all predicted stop positions are used to train the prediction model to obtain the trained prediction model, and the trained prediction model is used to predict the stop position of the vehicle to be predicted to obtain the final predicted stop position of the vehicle to be predicted. Among them, the constructed heterogeneous information network includes three aspects of information: identification information, stop points, and points of interest attributes of the target vehicle, which improves the multimodal information richness of the heterogeneous information network. The embedding vector of the node obtained based on the information-rich heterogeneous information network mines the correlation between the attributes of the target vehicle, stop points, and points of interest, improves the accuracy and comprehensiveness of the embedding vector, and predicts the stop location based on accurate embedding vectors, weather information, and stop point information. The information of embedded vectors, weather information, and stop point information is integrated to improve the information richness and accuracy of the data during prediction, thereby improving the accuracy of vehicle stop location prediction.

[0062] Next, the multi-modal vehicle stop position prediction method provided in this application is exemplified.

[0063] like Figure 1 As shown, the vehicle stop position prediction method based on multi-modality provided in this application includes the following steps:

[0064] Step 11, obtaining weather information of the target area at multiple historical moments, and the trajectory points of each target vehicle in the target area at all historical moments, and based on all the trajectory points of each target vehicle, obtaining multiple stop points of each target vehicle and the stop point information of each stop point.

[0065] A stop point is a location point where the vehicle moves less than a distance difference threshold (such as 10 meters) within a period of time (such as half an hour). The above-mentioned stop point information includes the stop time, stop time, location and corresponding interest point attributes of the stop point (i.e., the stop time, stop time, location of the target vehicle at the stop point, and the interest point attributes corresponding to the stop point). The above-mentioned target area can be a city, a county, etc., and the target vehicle is a vehicle with trajectory point data at multiple historical moments. All historical moments are moments before the current moment. For example, if the current moment is 9 o'clock, the historical moments can be 7 o'clock, 7:30, 8 o'clock, etc. The interest point attributes are the attributes of the urban facilities closest to the stop point, such as hospitals, parks, schools, shopping malls, etc.

[0066] In some embodiments of the present application, the track points of the target vehicle can be obtained through the global positioning system, and the weather information of the target area can be obtained by accessing a website that discloses the weather information of the target area. The above steps of obtaining multiple stop points of each target vehicle and the stop point information of each stop point based on all the track points of each target vehicle are specifically as follows:

[0067] For each target vehicle, perform the following steps:

[0068] In the first step, the distance difference between every two trajectory points of the target vehicle is calculated.

[0069] Specifically, through the formula:

[0070]

[0071] Calculate the distance difference d between the nth trajectory point and the mth trajectory point nm .

[0072] Where r is the radius of the earth, represents the latitude of the mth trajectory point, represents the latitude of the nth trajectory point, λ m represents the longitude of the mth trajectory point, λ n Represents the longitude of the nth trajectory point, n,m∈N,n≠m, N represents the set of trajectory point numbers of the target vehicle.

[0073] In the second step, for each distance difference, if the distance difference is less than the distance difference threshold, a stop point is generated, and the average longitude and latitude between the two trajectory points corresponding to the distance difference is used as the position of the stop point, the time difference between the historical moments corresponding to the two trajectory points is used as the stop duration of the stop point, and the earliest historical moment between the two trajectory points is used as the stop time of the stop point, and the attributes of the point of interest closest to the stop point are obtained.

[0074] For example, the distance difference between the first trajectory point of the target vehicle at 7 o'clock and the second trajectory point at 7:30 is 5 meters, which is less than the distance difference threshold of 10 meters. A stop point is generated. The position of the stop point is the average longitude and latitude of the first trajectory point and the second trajectory point. The stop duration is half an hour and the stop time is 7 o'clock. By querying the map or other methods, it is obtained that the nearest urban facility to the stop point is a school. Then the point of interest attribute of the stop point is school.

[0075] In the third step, the duration, time, location and corresponding POI attributes of each stop point are integrated to obtain the stop point information of the stop point.

[0076] Exemplarily, the stay duration, stay time, location and corresponding interest point attributes corresponding to the stay point are integrated into one piece of data to obtain the stay point information of the stay point.

[0077] It is worth mentioning that this step can accurately obtain the stay points in space. At the same time, the stay point information includes four aspects of information: stay duration, stay time, location, and point of interest attributes, which can effectively improve the richness of the multimodal data of the stay point information.

[0078] Step 12: construct a heterogeneous information network based on all target vehicles, all stop points, all stop point information, and all points of interest attributes.

[0079] The attribute of each node in the above heterogeneous information network is one of the identification information of multiple target vehicles, multiple stop points, and multiple points of interest attributes, and all the identification information of all target vehicles, all stop points, and all the points of interest attributes have a corresponding node, and the edge between the nodes is the association relationship between the attributes of the corresponding two nodes. The identification information of the target vehicle is used to identify the target vehicle, such as the serial number and license plate number of the target vehicle.

[0080] It should be noted that the identification information of all target vehicles, all stopovers, and all points of interest attributes all have corresponding nodes in the heterogeneous information network, that is, there are three types of nodes in the heterogeneous information network: nodes corresponding to the identification information of target vehicles, nodes corresponding to stopovers, and nodes corresponding to the attributes of points of interest. If the target vehicle c has visited stopover point i, an edge is generated between the node corresponding to the identification information of the target vehicle c and the node corresponding to stopover point i. If the point of interest attribute of stopover point i is the same as the point of interest attribute a, an edge is generated between the node corresponding to stopover point i and the node corresponding to the point of interest attribute a. In addition, if there are two or more stopovers with the same location and two or more points of interest with the same attributes, only one node is retained. For example, if the point of interest attribute of the first stopover point is hospital, and the point of interest attribute of the second stopover point is also hospital, only one node corresponding to the hospital is retained in the heterogeneous information network.

[0081] It is worth mentioning that by constructing a heterogeneous information network, the identification information, stop points, and correlation relationships between points of interest of the target vehicle can be integrated into the same network, thereby improving the information richness of the heterogeneous information network.

[0082] Step 13, mapping the heterogeneous information network to obtain the embedding vector of each node in the heterogeneous information network.

[0083] Specifically, through the formula:

[0084]

[0085] Calculate the embedding vector value of the i-th node at the x+1 layer

[0086] in, represents the embedding vector value of the i-th node in the x-th layer, and Both represent learnable parameters, represents the mapping embedding vector of the i-th node mapped to the j-th node in the x+1-th layer, represents the parameter training matrix, N i represents the set of all adjacent nodes of the ith node in the heterogeneous information network, v j represents the jth node in the heterogeneous information network, v i represents the i-th node in a heterogeneous information network, represents the correlation coefficient between the i-th node and the j-th node, θ represents the trainable parameter, σ represents the activation function, i, j∈M, i≠j, M represents the set of node numbers in the heterogeneous information network, || represents the connection operation, x=0,1,...,X-1, X represents the last layer of the mapping, when x=0, are all initial vector values ​​(as the input of the 0th layer mapping, the embedding vector value of the 1st layer is calculated by the above formula). When x=X-1, the embedding vector value of the i-th node in the Xth layer is As the embedding vector of the i-th node (that is, when mapped to the last layer, the embedding vector value corresponding to the last layer is used as the embedding vector of the node that needs to be calculated in this step).

[0087] Exemplarily, the one-hot encoding technique may be used to obtain the initial vector value of each node, and a multi-layer mapping neural network may be used to perform this step to obtain the embedding vector of each node.

[0088] It is worth mentioning that by mapping heterogeneous information networks, the correlation between nodes can be mined, which improves the accuracy of embedding vectors and the effect of multimodal data interaction.

[0089] Step 14: for each target vehicle, respectively, concatenate the embedding vector corresponding to each stop point of the target vehicle with the embedding vector corresponding to the identification information of the target vehicle to obtain multiple stop point vectors, and combine all the stop point vectors into a vector trajectory.

[0090] In some embodiments of the present application, the above step of combining all the stay point vectors into a vector trajectory is specifically: sorting all the stay point vectors in order of the stay times corresponding to all the stay point vectors from early to late to obtain the vector trajectory.

[0091] Exemplarily, the target vehicle has three stop points, and the corresponding stop moments are the first moment, the third moment, and the fourth moment. After the embedding vector corresponding to each stop point is concatenated with the embedding vector corresponding to the identification information of the target vehicle to obtain the stop point vector corresponding to the stop point, the three stop point vectors are sorted in the order of the first moment, the third moment, and the fourth moment to obtain the vector trajectory of the target vehicle.

[0092] It is worth mentioning that the embedded vector of the stop point and the embedded vector of the target vehicle are spliced ​​together to achieve the fusion of the information of the stop point and the target vehicle. By sorting the stop point vectors according to the stop time corresponding to the stop point, the vector trajectory has time sequence, which improves the effect of data fusion.

[0093] Step 15, based on all weather information, all stop point information, and all vector trajectories, a prediction model is used to predict the stop position of each target vehicle to obtain a predicted stop position of each target vehicle.

[0094] In some embodiments of the present application, the above-mentioned step of predicting the stop position of each target vehicle using a prediction model based on all weather information, all stop point information, and all vector trajectories to obtain the predicted stop position of each target vehicle is specifically as follows:

[0095] In the first step, for each stop point, the stop time and stop duration in the stop point information are embedded to obtain a time embedding vector, and the weather information corresponding to the stop time of the stop point is embedded to obtain a weather embedding vector.

[0096] Exemplarily, a fully connected network or a linear network may be used to perform an embedding operation to obtain a time embedding vector and a weather embedding vector.

[0097] In the second step, for each vector trajectory, each stop point vector in the vector trajectory is concatenated with the time embedding vector and weather embedding vector corresponding to the stop point vector to obtain the overall vector trajectory.

[0098] In the third step, the prediction model is used to calculate each overall vector trajectory to obtain the final vector trajectory corresponding to each target vehicle, and each final vector trajectory is predicted to obtain the predicted stop position of each target vehicle.

[0099] In some embodiments of the present application, the above-mentioned prediction model can be an attention neural network and a fully connected neural network connected in sequence, the input end of the attention neural network is the input end of the prediction model, and the output end of the fully connected neural network is the output end of the prediction model.

[0100] After each overall vector trajectory is input into the prediction model, the attention neural network calculates the overall vector trajectory through the attention mechanism, and uses different weight matrices to map the overall vector trajectory to the query (Q), key (K), and value (V) vectors, and then uses the dot product of Q and K to calculate the attention score, and then scales it by a scaling factor (usually the square root of the dimension of K), normalizes the scaled attention score through the softmax function, and obtains the attention weight. The attention weight is used to perform weighted summation on V to obtain the weighted output vector (i.e., the final vector trajectory), and then predicted by the fully connected neural network to obtain the predicted stop position. The specific process of predicting each final vector trajectory and obtaining the predicted stop position of each target vehicle is as follows:

[0101] By formula:

[0102] l = tanh(W o ×h+b o )

[0103] Calculate the predicted stop position l of the target vehicle.

[0104] Where h represents the final vector trajectory of the target vehicle, W o and b o are all learnable parameters.

[0105] It should be noted that the above-mentioned predicted stop position is the stop position of the target vehicle at a time after the time period corresponding to the historical trajectory.

[0106] It can be understood that the above formula for calculating the predicted stop position of the target vehicle is an expression of a fully connected neural network.

[0107] It is worth mentioning that by predicting the stop location based on accurate embedding vectors, weather information and stop point information, the information of embedding vectors, weather information and stop point information is integrated to improve the information richness and accuracy of the data during prediction, thereby improving the accuracy of vehicle stop location prediction.

[0108] Step 16, using all predicted stop positions to train the prediction model to obtain a trained prediction model, and using the trained prediction model to predict the stop position of the vehicle to be predicted to obtain the final predicted stop position of the vehicle to be predicted.

[0109] The above-mentioned vehicle to be predicted is a vehicle whose stop position needs to be predicted.

[0110] In some embodiments of the present application, the prediction model can be trained by constructing a loss function (such as a comparison loss function) based on the predicted stop position and the actual stop point position of the target vehicle after the time period corresponding to the historical trajectory. For example, only when the value of the loss function is less than a preset value, the prediction model training is considered to be completed, and it is a trained prediction model. When the value of the loss function is greater than or equal to the preset value, the parameters of the prediction model are adjusted. The above-mentioned use of the trained prediction model to predict the stop position of the vehicle to be predicted to obtain the final predicted stop position of the vehicle to be predicted is specifically as follows: after obtaining multiple stop points and stop point information of the vehicle to be predicted, the vehicle to be predicted and all stop points are added to the heterogeneous information network, and the overall vector trajectory of the vehicle to be predicted is obtained according to the processing method of steps 13 to 15, and the stop position prediction of the overall vector trajectory of the vehicle to be predicted is performed using the trained prediction model to obtain the final predicted stop position. That is, the prediction of the future stop position is achieved through the stop positions at multiple historical moments.

[0111] The multi-modal vehicle stop position prediction method of the present application is exemplified below with reference to a specific example.

[0112] like Figure 2As shown in the figure, by analyzing the original global positioning system (GPS) trajectory, the stop points are obtained, and a heterogeneous information network is constructed based on the stop points, vehicle ID (i.e., the identification information of the target vehicle), and POI information (i.e., the attributes of the point of interest). The embedding vector of each node is obtained based on the heterogeneous information network, and the vector trajectory is obtained according to the embedding vector and the timing information in the GPS trajectory. The vector trajectory is input into the prediction model together with the weather information and the time information to obtain the predicted value of the future stop point (i.e., the predicted stop position mentioned above).

[0113] It is worth mentioning that the constructed heterogeneous information network includes three aspects of information: identification information, stop points, and points of interest attributes of the target vehicle, which improves the multimodal information richness of the heterogeneous information network. The embedding vector of the node obtained based on the information-rich heterogeneous information network mines the correlation between the attributes of the target vehicle, stop points, and points of interest, improves the accuracy and comprehensiveness of the embedding vector, and predicts the stop location based on the accurate embedding vector, weather information, and stop point information. The information of the embedded vector, weather information, and stop point information is integrated to improve the information richness and accuracy of the data during prediction, thereby improving the accuracy of the vehicle stop location prediction.

[0114] The following is an exemplary description of the multi-modal based vehicle stop position prediction device provided in the present application.

[0115] like Figure 3 As shown, the embodiment of the present application provides a vehicle stop position prediction device based on multi-modality, and the vehicle stop position prediction device 300 based on multi-modality includes:

[0116] The acquisition module 301 acquires weather information of the target area at multiple historical moments, and the track points of each target vehicle in the target area at all historical moments, and acquires multiple stop points of each target vehicle and the stop point information of each stop point based on all the track points of each target vehicle; the stop point information includes the stop time, stop time, location and corresponding interest point attributes of the stop point;

[0117] A construction module 302 constructs a heterogeneous information network according to all target vehicles, all stop points, all stop point information, and all interest point attributes; the attribute of each node in the heterogeneous information network is one of the identification information of multiple target vehicles, multiple stop points, and multiple interest point attributes, and the identification information of all target vehicles, all stop points, and all interest point attributes have a corresponding node, and the edge between the nodes is the association relationship between the attributes of the corresponding two nodes;

[0118] A mapping module 303 maps the heterogeneous information network to obtain an embedding vector of each node in the heterogeneous information network;

[0119] A concatenation module 304 concatenates, for each target vehicle, the embedded vector corresponding to each stop point of the target vehicle with the embedded vector corresponding to the identification information of the target vehicle to obtain a plurality of stop point vectors, and combines all the stop point vectors into a vector trajectory;

[0120] The stop position prediction module 305 predicts the stop position of each target vehicle using a prediction model based on all weather information, all stop point information, and all vector trajectories to obtain a predicted stop position of each target vehicle;

[0121] The training module 306 trains the prediction model using all predicted stop positions to obtain a trained prediction model, and uses the trained prediction model to predict the stop position of the vehicle to be predicted to obtain the final predicted stop position of the vehicle to be predicted.

[0122] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0123] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0124] like Figure 4 As shown, an embodiment of the present application provides a terminal device, and the terminal device D10 of this embodiment includes: at least one processor D100 ( Figure 4 Only one processor is shown in the figure), a memory D101, and a computer program D102 stored in the memory D101 and executable on the at least one processor D100, wherein the processor D100 implements the steps of any of the above-mentioned method embodiments when executing the computer program D102.

[0125] Specifically, when the processor D100 executes the computer program D102, it obtains multiple stop points of each target vehicle and the stop point information of each stop point based on all trajectory points of each target vehicle, then constructs a heterogeneous information network based on all target vehicles, all stop points, all stop point information, and all interest point attributes, and then maps the heterogeneous information network to obtain the embedding vector of each node in the heterogeneous information network, and then, for each target vehicle, respectively, concatenates the embedding vector corresponding to each stop point of the target vehicle with the embedding vector corresponding to the identification information of the target vehicle to obtain multiple stop point vectors, and combines all stop point vectors into vector trajectories, and then, based on all weather information, all stop point information, and all vector trajectories, uses a prediction model to predict the stop position of each target vehicle to obtain a predicted stop position of each target vehicle, and finally uses all predicted stop positions to train the prediction model to obtain a trained prediction model, and uses the trained prediction model to predict the stop position of the vehicle to be predicted to obtain the final predicted stop position of the vehicle to be predicted. Among them, the constructed heterogeneous information network includes three aspects of information: identification information, stop points, and points of interest attributes of the target vehicle, which improves the multimodal information richness of the heterogeneous information network. The embedding vector of the node obtained based on the information-rich heterogeneous information network mines the correlation between the attributes of the target vehicle, stop points, and points of interest, improves the accuracy and comprehensiveness of the embedding vector, and predicts the stop location based on accurate embedding vectors, weather information, and stop point information. The information of embedded vectors, weather information, and stop point information is integrated to improve the information richness and accuracy of the data during prediction, thereby improving the accuracy of vehicle stop location prediction.

[0126] The processor D100 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0127] In some embodiments, the memory D101 may be an internal storage unit of the terminal device D10, such as a hard disk or memory of the terminal device D10. In other embodiments, the memory D101 may also be an external storage device of the terminal device D10, such as a plug-in hard disk, a smart memory card (SMC, SmartMedia Card), a secure digital (SD, Secure Digital) card, a flash card (Flash Card), etc. equipped on the terminal device D10. Further, the memory D101 may also include both an internal storage unit of the terminal device D10 and an external storage device. The memory D101 is used to store an operating system, an application program, a boot loader (BootLoader), data and other programs, such as the program code of the computer program, etc. The memory D101 may also be used to temporarily store data that has been output or is to be output.

[0128] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0129] An embodiment of the present application provides a computer program product. When the computer program product is run on a terminal device, the terminal device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0130] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), electric carrier signal, telecommunication signal and software distribution medium that can carry the computer program code to the multi-modal vehicle stop position prediction method device / terminal device. For example, a USB flash drive, a mobile hard disk, a disk or an optical disk.

[0131] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0132] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0133] The above is a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles described in the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A vehicle stop position prediction method based on multi-modality, characterized in that: include: Acquire weather information of a target area at multiple historical moments, and the track points of each target vehicle in the target area at all historical moments, and acquire multiple stop points of each target vehicle and the stop point information of each stop point based on all the track points of each target vehicle; the stop point information includes the stop time, stop time, location and corresponding interest point attributes of the stop point; A heterogeneous information network is constructed according to all target vehicles, all stop points, all stop point information, and all interest point attributes; the attribute of each node in the heterogeneous information network is one of the identification information of multiple target vehicles, multiple stop points, and multiple interest point attributes, and the identification information of all target vehicles, all stop points, and all interest point attributes have a corresponding node, and the edge between the nodes is the association relationship between the attributes of the corresponding two nodes; Mapping the heterogeneous information network to obtain an embedding vector of each node in the heterogeneous information network; For each of the target vehicles, respectively, concatenate the embedding vector corresponding to each stop point of the target vehicle with the embedding vector corresponding to the identification information of the target vehicle to obtain multiple stop point vectors, and combine all the stop point vectors into a vector trajectory; Based on all weather information, all stop point information, and all vector trajectories, a prediction model is used to predict the stop position of each target vehicle to obtain a predicted stop position of each target vehicle; The prediction model is trained using all predicted stop positions to obtain a trained prediction model, and the trained prediction model is used to predict the stop position of the vehicle to be predicted to obtain the final predicted stop position of the vehicle to be predicted.

2. The vehicle stop position prediction method according to claim 1, characterized in that: The step of acquiring multiple stop points of each target vehicle and stop point information of each stop point based on all trajectory points of each target vehicle includes: For each target vehicle, perform the following steps: Calculating the distance difference between every two trajectory points of the target vehicle; For each of the distance differences, if the distance difference is less than the distance difference threshold, a stop point is generated, and the average longitude and latitude between the two trajectory points corresponding to the distance difference is used as the position of the stop point, the time difference between the historical moments corresponding to the two trajectory points is used as the stop duration of the stop point, and the earliest historical moment of the historical moments corresponding to the two trajectory points is used as the stop time of the stop point, and the attribute of the point of interest closest to the stop point is obtained; The stay duration, stay time, location and corresponding point of interest attributes of each stay point are integrated to obtain the stay point information of the stay point.

3. The vehicle stop position prediction method according to claim 2, characterized in that: The calculating the distance difference between every two trajectory points of the target vehicle comprises: By formula: Calculate the distance difference d between the nth trajectory point and the mth trajectory point nm ; Where r is the radius of the earth, represents the latitude of the mth trajectory point, represents the latitude of the nth trajectory point, λ m represents the longitude of the mth trajectory point, λ n represents the longitude of the nth trajectory point, n,m∈N,n≠m, and N represents the set of trajectory point serial numbers of the target vehicle.

4. The vehicle stop position prediction method according to claim 1, characterized in that: Mapping the heterogeneous information network to obtain an embedding vector of each node in the heterogeneous information network includes: By formula: Calculate the embedding vector value of the i-th node at the x+1 layer in, represents the embedding vector value of the i-th node in the x-th layer, and Both represent learnable parameters, represents the mapping embedding vector of the i-th node mapped to the j-th node in the x+1-th layer, represents the parameter training matrix, N i represents the set of all adjacent nodes of the ith node in the heterogeneous information network, v j represents the jth node in the heterogeneous information network, v i represents the i-th node in a heterogeneous information network, represents the correlation coefficient between the i-th node and the j-th node, θ represents the trainable parameter, σ represents the activation function, i, j∈M, i≠j, M represents the set of node numbers in the heterogeneous information network, || represents the connection operation, x=0,1,...,X-1, X represents the last layer of the mapping, when x=0, are all initial vectors. When x=X-1, the embedding vector value of the i-th node in the X-th layer is as the embedding vector of the i-th node.

5. The vehicle stop position prediction method according to claim 1, characterized in that: The step of combining all stop point vectors into a vector trajectory includes: All the stop point vectors are sorted according to the order of the stop times corresponding to them from early to late to obtain the vector trajectory.

6. The vehicle stop position prediction method according to claim 1, characterized in that: The method of predicting the stop position of each target vehicle based on all weather information, all stop point information, and all vector trajectories using a prediction model to obtain a predicted stop position of each target vehicle includes: For each of the stop points, embedding operation is performed on the stop time and the stop duration in the stop point information of the stop point to obtain a time embedding vector, and embedding operation is performed on the weather information corresponding to the stop time of the stop point to obtain a weather embedding vector; For each of the vector trajectories, concatenate each stop point vector in the vector trajectory with the time embedding vector and the weather embedding vector corresponding to the stop point vector to obtain an overall vector trajectory; The prediction model is used to calculate each overall vector trajectory to obtain a final vector trajectory corresponding to each target vehicle, and each final vector trajectory is predicted to obtain a predicted stop position of each target vehicle.

7. The vehicle stop position prediction method according to claim 6, characterized in that: The step of predicting each of the final vector trajectories to obtain a predicted stop position of each of the target vehicles includes: By formula: l=tanh(W o ×h+b o ) Calculate the predicted stop position l of the target vehicle; Where h represents the final vector trajectory of the target vehicle, W o and b o All are parameters.

8. A vehicle stop position prediction device based on multi-modality, characterized in that: include: An acquisition module is used to acquire weather information of a target area at multiple historical moments, and track points of each target vehicle in the target area at all historical moments, and based on all track points of each target vehicle, acquire multiple stop points of each target vehicle and stop point information of each stop point; the stop point information includes the stop time, stop time, location and corresponding interest point attributes of the stop point; A construction module is provided to construct a heterogeneous information network according to all target vehicles, all stop points, all stop point information, and all interest point attributes; the attribute of each node in the heterogeneous information network is one of the identification information of multiple target vehicles, multiple stop points, and multiple interest point attributes, and the identification information of all target vehicles, all stop points, and all interest point attributes have a corresponding node, and the edge between the nodes is the association relationship between the attributes of the corresponding two nodes; A mapping module maps the heterogeneous information network to obtain an embedding vector of each node in the heterogeneous information network; A splicing module, for each of the target vehicles, splices the embedded vector corresponding to each stop point of the target vehicle with the embedded vector corresponding to the identification information of the target vehicle to obtain multiple stop point vectors, and combines all the stop point vectors into a vector trajectory; A stop position prediction module, based on all weather information, all stop point information, and all vector trajectories, uses a prediction model to predict the stop position of each target vehicle to obtain a predicted stop position of each target vehicle; The training module uses all predicted stop positions to train the prediction model to obtain a trained prediction model, and uses the trained prediction model to predict the stop position of the vehicle to be predicted to obtain the final predicted stop position of the vehicle to be predicted.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the vehicle stop position prediction method based on multimodality is implemented as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the vehicle stop position prediction method based on multi-modality is implemented as described in any one of claims 1 to 7.

Images