Transportation travel order prediction method, device and equipment

Abstract

The present application provides a method, device and equipment for predicting transportation orders, the method comprising: obtaining transportation orders for multiple regions of a target city in each unit time period within a first time period to obtain multiple first historical order sequences; extracting spatial feature vectors from the first historical order sequences in multiple regions through a self-attention neural network model; extracting time feature vectors from the first historical order sequences in each unit time period through a self-attention neural network model; predicting transportation orders for at least one region in one or more future unit time periods based on the time feature vector and the spatial feature vector through the self-attention neural network model. The present application extracts spatial feature vectors and time feature vectors from the historical order sequence, and then combines the two to predict future transportation orders, thereby changing the number of model calculations from full to sparse, thereby improving the model calculation efficiency.

Description

Technical Field

[0001] The present application belongs to the field of transportation technology, and in particular, relates to a method, device and equipment for predicting transportation travel orders. Background Art

[0002] In the field of transportation, historically recorded transportation order forecasts play a vital role in traffic light control, vehicle dispatching, service pricing, and risk management. For example, in a shared bike or taxi system, transportation order forecasts can help operators dispatch bikes / taxi from areas with oversupply to areas with insufficient supply. The effective pricing schemes of current ride-hailing platforms rely heavily on accurate transportation order forecasts.

[0003] At present, the technologies for predicting transportation orders mainly include traditional models, data mining, and deep learning. Although traditional models such as time series models and regression analysis fully connected neural network models can use historical data for prediction, they have limitations in dealing with complex spatiotemporal cross-dependencies, are difficult to handle large-scale data, and have limited prediction accuracy. Data mining technology can extract patterns from a large amount of historical data, but the data dependency is too strong. If the data is noisy, missing, or incomplete, it will affect the accuracy of the prediction results.

[0004] In recent years, deep learning has been widely used in the field of transportation order prediction, especially deep learning models based on self-attention neural networks. However, when processing large-scale transportation orders, existing deep learning models based on self-attention neural networks perform full self-attention calculations on the historical order sequences formed by transportation orders, which results in huge computational complexity and makes the model difficult to converge. Summary of the invention

[0005] The embodiments of the present application provide a method, device and equipment for predicting transportation orders, which can solve the problem that the existing self-attention neural network model performs full self-attention calculation on the historical order sequence formed by transportation orders when predicting transportation orders, which results in huge calculation amount and the model is not easy to converge.

[0006] In a first aspect, an embodiment of the present application provides a method for predicting a transportation order, comprising:

[0007] Obtaining the transportation travel orders of multiple areas of the target city in each unit time period during the first time period, and obtaining multiple first historical order sequences;

[0008] Classifying the plurality of first historical order sequences by the region, and extracting spatial feature vectors from the first historical order sequences in vector form within the plurality of the regions through a self-attention neural network model;

[0009] Classifying the plurality of first historical order sequences by the unit time, and extracting a time feature vector from the first historical order sequence in vector form within each of the unit time periods by using the self-attention neural network model;

[0010] The self-attention neural network model is used to predict the transportation travel orders in at least one region in one or more future unit time periods based on the time feature vector and the space feature vector.

[0011] In a possible implementation manner of the first aspect, the self-attention neural network model includes an encoder, wherein the encoder includes a first multi-head self-attention layer and a second multi-head self-attention layer;

[0012] The plurality of first historical order sequences are classified according to the region, and spatial feature vectors are extracted from the first historical order sequences in vector form within the plurality of the regions through a self-attention neural network model, including:

[0013] Merging the first historical order sequences in the form of vectors in each of the regions to obtain a plurality of second historical order sequences;

[0014] The regions are grouped based on the density of travel between the regions to obtain a plurality of spatial groups; wherein the density of travel between the regions is determined according to at least one of the order quantity, travel time, and travel distance of the transportation travel orders per unit time;

[0015] Through the first multi-head self-attention layer, self-attention calculation is performed between the second historical order sequences in each of the spatial groups to extract local spatial feature vectors;

[0016] Merging all second historical order sequences in the space group to obtain multiple third historical order sequences;

[0017] Through the second multi-head self-attention layer, self-attention calculation is performed between each of the third historical order sequences to extract the global spatial feature vector.

[0018] In a possible implementation manner of the first aspect, the encoder further includes a third multi-head self-attention layer, a fourth multi-head self-attention layer, and a fifth multi-head self-attention layer;

[0019] Classifying the plurality of first historical order sequences by the unit time, and extracting a time feature vector from the first historical order sequence in vector form within each unit time period by the self-attention neural network model, comprises:

[0020] Merging the first historical order sequences in vector form within each of the unit time periods to obtain a plurality of fourth historical order sequences;

[0021] Through the third multi-head self-attention layer, self-attention calculation is performed between the fourth historical order sequences within each day to extract the time feature vector of the unit time period;

[0022] If the first time period is greater than one week, merging the fourth historical order sequences of each day to obtain multiple fifth historical order sequences;

[0023] Through the fourth multi-head self-attention layer, self-attention calculation is performed between the fifth historical order sequences within each week to extract the time feature vector of the day;

[0024] If the first time period is greater than or equal to one month, merging the fifth historical order sequences of each week to obtain multiple sixth historical order sequences;

[0025] Through the fifth multi-head self-attention layer, self-attention calculation is performed between the sixth historical order sequences within each month to extract the weekly time feature vector.

[0026] In a possible implementation of the first aspect, environmental information is also embedded in the third multi-head self-attention layer, so that the third multi-head self-attention layer also extracts the environmental information feature vector of each unit time period; the environmental information includes at least one of the weather, temperature, wind speed, and road condition information corresponding to each unit time period.

[0027] In a possible implementation manner of the first aspect, predicting the transportation travel orders of at least one region in one or more future unit time periods according to the time feature vector and the spatial feature vector by using the self-attention neural network model includes:

[0028] The local space feature vector, the global space feature vector, the time feature vector of a unit time period, the time feature vector of a day, the time feature vector of a week, and the environmental information feature vector output by the encoder are fused as the input of the decoder;

[0029] The decoder is used to predict the transportation travel order forecast value of at least one of the regions in one or more of the unit time periods.

[0030] In a possible implementation of the first aspect, the decoder includes a sixth multi-head self-attention layer and a seventh multi-head self-attention layer;

[0031] Before predicting the predicted value of the transportation order of at least one of the regions in one or more of the unit time periods by the decoder, the method further includes:

[0032] Get a preset number of placeholders;

[0033] By means of the sixth multi-head self-attention layer, a self-attention calculation is performed between the first historical order sequence in the form of a vector of at least one of the regions in the most recent one or more unit time periods and the placeholder, so that the placeholder obtains the spatiotemporal feature vector of at least one of the regions in the most recent one or more unit time periods;

[0034] Through the seventh multi-head self-attention layer, a cross-self-attention calculation is performed on the spatiotemporal feature vector of the placeholder and the fused feature vector of the seventh multi-head self-attention layer input into the decoder to update the spatiotemporal feature vector of the placeholder.

[0035] The predicting, by the decoder, the predicted value of the transportation order of at least one of the regions in one or more of the unit time periods comprises:

[0036] Outputting, by the decoder, a predicted order sequence of at least one of the regions in the preset number of the unit time periods in the placeholders corresponding to the preset number of the placeholders;

[0037] According to the predicted order sequence, the predicted transportation orders are obtained.

[0038] In a possible implementation manner of the first aspect, the classifying the plurality of first historical order sequences by the region, and extracting spatial feature vectors from the first historical order sequences in vector form within the plurality of regions by a self-attention neural network model, includes:

[0039] Merging the first historical order sequences in the form of vectors in each of the regions to obtain a plurality of second historical order sequences;

[0040] By using the self-attention neural network model, performing self-attention calculation between the second historical order sequences to extract the spatial feature vector;

[0041] Classifying the plurality of first historical order sequences by the unit time, and extracting a time feature vector from the first historical order sequence in vector form within each unit time period by using the self-attention neural network model, including:

[0042] Merging the first historical order sequences in vector form within each of the unit time periods to obtain a plurality of fourth historical order sequences;

[0043] Through the self-attention neural network model, self-attention calculation is performed between the fourth historical order sequence to extract the time feature vector.

[0044] In a possible implementation manner of the first aspect, after obtaining the plurality of first historical order sequences, the method further includes:

[0045] Preprocessing the plurality of first historical order sequences;

[0046] According to the pre-processed plurality of first historical order sequences, a topological graph network corresponding to each unit time is constructed; wherein each region represents a node in the topological graph network, and if two regions are the starting point and destination of the transportation orders for each other, and the number of transportation orders in one or more unit time periods is greater than a preset threshold, a connection is established between the two regions, and the weight of the connection is determined according to at least one of the number of transportation orders, travel time, and travel distance in a unit time; and the weight of the connection is used as the closeness of the exchanges between the regions;

[0047] The topological graph network is converted into a vector form to obtain the first historical order sequence in a vector form.

[0048] In a second aspect, an embodiment of the present application provides a transportation order prediction device, comprising:

[0049] An acquisition module is used to acquire the transportation travel orders of multiple areas of the target city in each unit time period during the first time period to obtain multiple first historical order sequences;

[0050] a spatial feature extraction module, configured to classify the plurality of first historical order sequences by the region, and extract spatial feature vectors from the first historical order sequences in vector form within the plurality of the regions through a self-attention neural network model;

[0051] A time feature extraction module, used to classify the plurality of first historical order sequences according to the unit time, and extract a time feature vector from the first historical order sequence in vector form within each of the unit time periods through the self-attention neural network model;

[0052] A prediction module is used to predict the transportation travel orders of at least one region in one or more unit time periods in the future according to the time feature vector and the spatial feature vector through the self-attention neural network model.

[0053] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements a method as described in any one of the first aspects when executing the computer program.

[0054] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of the first aspects is implemented.

[0055] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes any method described in the first aspect.

[0056] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.

[0057] Compared with the prior art, the embodiments of the present application have the following beneficial effects: the embodiments of the present application simplify and decompose the complex spatiotemporal cross-dependencies in the problem of traffic order prediction into spatial dependencies and temporal dependencies, and then extract spatial feature vectors and temporal feature vectors respectively, and then combine the spatial feature vectors and the temporal feature vectors to predict future traffic orders, thereby reducing the number of calculations of the self-attention neural network model from full to sparse, improving the model calculation efficiency, and also making the model easier to converge. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0059] Figure 1 This is a flow chart of a method for predicting transportation orders provided by an embodiment of the present application. Figure 1 ;

[0060] Figure 2 This is a framework diagram of a self-attention neural network model provided by an embodiment of the present application;

[0061] Figure 3 This is a flow chart of a method for predicting transportation orders provided by an embodiment of the present application. Figure 2 ;

[0062] Figure 4 This is a flow chart of a method for predicting transportation orders provided by an embodiment of the present application. Figure 3 ;

[0063] Figure 5 It is a visualized heat map of the traffic orders of the target city in the next 3 hours predicted according to the traffic order prediction method provided in one embodiment of the present application;

[0064] Figure 6 It is a structural block diagram of a transportation order prediction device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0065] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0066] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0067] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0068] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0069] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0070] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0071] To facilitate understanding of the following introduction to the transportation travel order prediction method provided in the embodiment of the present application, some terms used in the embodiment of the present application are explained below for easier understanding.

[0072] Glossary:

[0073] Self-Attention Mechanism: The Self-Attention Mechanism is a method for processing sequence data in a self-attention neural network model, which is used to dynamically weight each element in the input sequence. It automatically assigns weights to different elements by calculating the correlation (attention weight) between elements in the input sequence, so that the self-attention neural network model can better capture the long-term dependencies in the sequence.

[0074] Multi-Head Self-Attention: Multi-Head Self-Attention is an extension of the self-attention mechanism. It adds multiple "heads" based on the original self-attention mechanism. Different heads can focus on different parts or different features of the sequence, thereby capturing different aspects of information in the sequence data. The advantage of this is that it can improve the expressiveness of the model because it can learn multiple features of the sequence data in parallel.

[0075] Spatial dependency: Spatial dependency means that in geographic space, the number of transportation orders in certain areas will be affected by neighboring areas. Because transportation demand is often affected by multiple factors such as traffic conditions, population density, and commercial activities in the surrounding areas. For example, the number of transportation orders in a bustling commercial district may be affected by the travel demand of adjacent residential and office areas. In addition, travel demand is usually higher near transportation facilities such as bus stops, subway stations, airports, or train stations. Therefore, when predicting travel orders at a certain location, it is necessary to consider the characteristics of its surrounding environment and neighboring locations. By analyzing time dependency, we can know the mutual influence between different regions.

[0076] Time dependency: Time dependency refers to the correlation of transportation orders in the time dimension. Specifically, the travel demand at a certain moment is often related to the demand at the previous moment. For example, the number of transportation orders during the morning rush hour is usually affected by the number of transportation orders in the previous hour or hours. In the transportation order forecast, this means that today's travel pattern may be similar to yesterday, and tomorrow's travel pattern may be similar to today. This pattern is reflected in different times of the day (morning rush hour, evening rush hour), different days of the week (weekdays, weekends), different seasons and even holidays. Understanding these periodic and trend changes is crucial to accurately predict future travel orders. Therefore, by analyzing time dependency, we can know the trend or pattern of transportation orders changing over time.

[0077] Spatiotemporal cross-dependency: Spatiotemporal cross-dependency involves two dimensions, time and space, and refers to the mutual influence and dependence between time and space. This means that changes in time will affect the state of space, and vice versa. For example, traffic congestion in one area may lead to an increase in the number of orders in a neighboring area, and this effect may persist for several hours in the future.

[0078] Figure 1 A schematic flow chart of a method for predicting a transportation order provided by an embodiment of the present application is shown. Figure 2 The framework diagram of the self-attention neural network model provided by the embodiment of the present application is shown. The method includes:

[0079] S110: Obtaining transportation travel orders for multiple areas of the target city in each unit time period within a first time period to obtain multiple first historical order sequences.

[0080] For example, the first time period may be one month, one quarter, one year or others. The unit time period may be one hour, one half hour, one 15 minutes or others. The regions of the target city may be divided by longitude and latitude, or by communities, streets or the like.

[0081] The start and end time of the "first time period" should be understood as the first time period from now on, rather than the first time period a period of time (such as a day / week / month, etc.) ago. "Multiple regions" can be the areas covered by the central urban area of ​​the target city, because the areas covered by the central urban area account for the largest proportion of transportation orders; or it can be all areas of the target city.

[0082] Collect the traffic order data of multiple areas in the target city in each unit time period during the first time period. These traffic order data can be obtained through channels such as urban traffic management systems, taxi companies, online car-hailing platforms, and shared bicycle platforms.

[0083] After the transportation order data is collected, all transportation orders within each unit time period in multiple regions can be arranged in chronological order to form multiple first historical order sequences.

[0084] Transportation orders include pick-up orders and / or disembarkation orders. That is, a transportation order is counted once when getting on the bus, and a pick-up order is obtained; it is counted once when getting off the bus, and a disembarkation order is obtained. The pick-up order and disembarkation order may include at least one of the following information: pick-up time (i.e., order start time), pick-up location / area to which the pick-up location belongs, disembarkation time (i.e., order end time), disembarkation location / number of areas to which the disembarkation location belongs, travel distance, travel time (i.e., the time between the order start and the order end), fare, number of passengers, etc.

[0085] The first historical order sequence is a sequence formed by all transportation orders in a certain unit time period in a certain area, that is, all transportation orders in a unit time period in a certain area correspond to a first historical order sequence.

[0086] The first historical order sequence may include the following information: region identifier, timestamp, and a number of transportation orders. From the number of transportation orders, the number of transportation orders (including the number of pickup orders and / or the number of drop-off orders), the average travel time, average travel distance, average number of passengers, average cost, and other information of each order may be analyzed.

[0087] For example, the unit time period is one hour, the target city is City A, City A has a total of 300 regions, and in the past month (30 days) in City A, the transportation orders of each region are counted once an hour. There are 24*30=720 hours in a month, so a total of 720*300=216,000 times are counted, that is, there are 720*300=216,000 first historical order sequences. The 720*300 first historical order sequences are used as a sample to predict the transportation orders of at least one region in one or more unit time periods in the future.

[0088] The traditional self-attention neural network model considers the spatiotemporal cross-dependency in the prediction of transportation orders, which means that the mutual influence between time and space factors needs to be considered, so full self-attention calculation is required. Full self-attention calculation means performing self-attention calculation between the first historical order sequences of each unit time period in all regions. For example, after obtaining 216,000 first historical order sequences, full self-attention calculation is performed on the 216,000 first historical order sequences, that is, (300*720) 2 =216000 2 The calculation is very large and the model is not easy to converge.

[0089] S120: Classify the plurality of first historical order sequences by region, and extract spatial feature vectors from the first historical order sequences in vector form within the plurality of regions through a self-attention neural network model.

[0090] Among them, the self-attention neural network model is obtained by training the initial self-attention neural network model through the historical order sequence.

[0091] The first historical order sequence in vector form is input into the self-attention neural network model. Based on the self-attention mechanism, the self-attention neural network model weights the first historical order sequences in different regions and extracts the spatial feature vector corresponding to each region. This spatial feature vector can represent the spatial dependency between different regions and the changing trend of the order quantity in different regions.

[0092] Based on the spatial dependence between different regions, we can analyze which regions have more orders (including pickup and drop-off orders), which regions have fewer orders, and which regions have close exchanges. Based on the changing trend of the number of orders in different regions, we can predict the number of orders, order amount, average travel distance per order, average time per order, and other information in a certain region or all regions in the future.

[0093] This embodiment divides the first historical order sequence into regions, does not consider the impact of time factors on each region, but only considers the mutual impact between different regions, and then extracts spatial feature vectors from the first historical order sequence in each region, which can greatly reduce the amount of calculation.

[0094] S130: Classify the plurality of first historical order sequences by unit time, and extract a time feature vector from the historical order sequence in vector form within each unit time period through a self-attention neural network model.

[0095] The first historical order sequence in vector form is input into the self-attention neural network model. The self-attention neural network model weights the first historical order sequence in different unit time periods through the self-attention mechanism and extracts the time feature vector corresponding to each unit time period. This time feature vector can represent the time dependency between different time periods and the change trend of the order quantity in different time periods.

[0096] According to the time dependency between different time periods, it is possible to analyze which unit time period has more orders (including pickup orders and drop-off orders) and which unit time periods have fewer orders. By analyzing the trend of order quantity changes in different time periods, it is possible to predict the order quantity, order amount, average travel distance per order, average time per order, and other information for a certain unit time period or all unit time periods in the future.

[0097] This embodiment divides the first historical order sequence into unit time, does not consider the impact of regional factors on each unit time period, but only considers the mutual impact between different unit time periods, and then extracts the time feature vector from the first historical order sequence in each unit time period, which can greatly reduce the amount of calculation.

[0098] S140: Predicting the transportation travel orders of at least one region in one or more future unit time periods according to the time feature vector and the space feature vector through a self-attention neural network model.

[0099] Since the spatial dependency between different regions and the changing trend of the number of orders in different regions can be known based on the spatial feature vector, and the temporal dependency between different time periods and the changing trend of the number of orders in different time periods can be known based on the temporal feature vector, it is possible to predict the transportation orders in a certain area or all areas in the next one or several hours.

[0100] It is worth noting that the transportation travel orders predicted in S140 may include at least one of the following information: the number of pick-up orders, the number of drop-off orders, the average cost per order / total cost, the average travel time per order, the average distance per order, the average number of passengers per order, etc. Exemplarily, the self-attention neural network model predicts the number of pick-up orders and the number of drop-off orders in all areas of the target city in the next 4 hours based on the time feature vector and the spatial feature vector.

[0101] The embodiment of the present application simplifies and decomposes the complex spatiotemporal cross-dependencies in the transportation order prediction problem into spatial dependencies and temporal dependencies, and then extracts the spatial feature vectors and the temporal feature vectors respectively, and then combines the spatial feature vectors and the temporal feature vectors to predict future transportation orders, thereby reducing the number of calculations of the self-attention neural network model from full to sparse, improving the model calculation efficiency and making the model easier to converge.

[0102] As an optional implementation in the embodiment of the present application, S110: after obtaining multiple first historical order sequences, it also includes S111-S113.

[0103] S111: Preprocessing a plurality of first historical order sequences.

[0104] Preprocessing can include: data analysis, data cleaning and data standardization.

[0105] Data analysis: By analyzing transportation orders, we can have a deeper understanding of the characteristics, rules and potential problems of transportation orders. Through data analysis, we can obtain: the scope of the data (such as whether the coordinates of the transportation order are within the operating scope of the taxi / online car-hailing / shared bicycle and the city, the longest and shortest travel time, the longest and shortest travel distance, location, etc.), central trend (such as the mean, median, mode and other statistical values ​​and distribution of travel time and travel distance), the closeness of exchanges between regions and possible outliers (such as invalid values ​​and missing values). This information can provide a strong basis and support for subsequent data processing, modeling and analysis, making the analysis results more accurate and reliable. For example, according to the central trend, it can be analyzed that there are more short distances, so that the weight of travel distance can be adjusted in S112. The scope of the data can directly affect the scores of travel distance, daily average travel time, and daily average number of orders in S112, and thus affect the connection weight between the two regions.

[0106] Data cleaning: processing invalid values ​​and missing values, etc. For example, missing values ​​or invalid values ​​caused by GPS positioning errors, the system not recording the starting point coordinates of the order, the same starting point coordinates, the starting point coordinates in other cities, and orders ending within 1 minute can be filled (such as using the mean, median, mode, etc.), or predicted and filled according to the algorithm, or the transportation orders with missing or invalid values ​​can be directly deleted.

[0107] Data standardization: Perform min-max standardization on the data to unify the dimensions of the values.

[0108] S112: Construct a topology network corresponding to each unit time according to the preprocessed plurality of first historical order sequences.

[0109] It is worth noting that the number of topology networks is the same as the number of unit time. For example, there are 720 hours in the past month (30 days), so 720 topology networks are constructed every hour.

[0110] When constructing a topological network, each region represents a node in the topological network. If two regions are each other's starting point and destination for transportation orders, and the number of transportation orders within a week is greater than a preset threshold (for example, 5), a connection is established between the two regions.

[0111] The connection weight is determined based on at least one of the travel time, travel distance, and number of transportation orders per unit time. If the number of transportation orders is less than 5 within a week, it means that the dependency (i.e., correlation) between the two regions can be ignored. For example, few people travel from the suburbs in the east of the city to the suburbs in the west of the city, and even if there are, they can be ignored. Ignoring these orders will not only not affect the accuracy of future order predictions, but will make the self-attention neural network model focus on more important information, which can improve the prediction accuracy of the self-attention neural network model.

[0112] The connection weight is the weight of the edge of the topological network. The travel time (which can be the average daily travel time or the total travel time of all transportation orders), travel distance (which can be the average daily travel distance or the total travel distance of all transportation orders), and the number of transportation orders per unit time are weighted to obtain the connection weight. Among them, the connection weight can be used as the closeness of exchanges between regions. Based on the above method, the topological network of all regions in each unit time period of the target city can be obtained.

[0113] Exemplarily, the connection weight is obtained by the following formula:

[0114] );

[0115] ;

[0116] ;

[0117] + ;

[0118] in, is the average daily travel distance, The average daily travel time, is the average daily order quantity, Indicates the region's indicators , , The score, is the weight score of the connection between two regions, subscript , Indicates the minimum and maximum values ​​of this indicator in all regions. , , It is a preset weight. The higher the score, the higher the average daily travel distance in the area. The higher the average daily travel time, The longer the average daily order quantity The more, the maximum is 1 and the minimum is 0. The higher the value, the closer the exchanges between regions.

[0119] S113: Convert the topological graph network into a vector form to obtain a first historical order sequence in a vector form.

[0120] Exemplarily, a unique identifier is assigned to each node, and an adjacency matrix or adjacency list is created to represent the topological graph network. The adjacency matrix is ​​a two-dimensional array whose rows and columns represent the identifiers of the nodes, respectively. If there is a connection between two nodes, a non-zero value (e.g., the weight of the connection) is set at the corresponding position; otherwise, it is set to zero. The adjacency list is a dictionary whose key is the identifier of the node and the value is a list of other nodes connected to the node. Traverse the first historical order sequence, and for each transportation order, check whether its origin and destination meet the preset threshold (e.g., the number of orders in a week is greater than 5). If the conditions are met, the corresponding connection information is updated in the adjacency matrix or adjacency list. The final adjacency matrix or adjacency list is the vector representation of the topological graph network.

[0121] As an optional implementation of an embodiment of the present application, S120: classify multiple first historical order sequences by region, and extract spatial feature vectors from the first historical order sequences in vector form within multiple regions through a self-attention neural network model, including S121-S122.

[0122] S121: Merge the first historical order sequences in vector form in each region to obtain multiple second historical order sequences.

[0123] S122: Perform self-attention calculation between the second historical order sequence through the self-attention neural network model to extract the spatial feature vector.

[0124] Combined with the above example, City A has 300 regions. Since there are 720 hours in the past month, each region has 720 first historical order sequences. The time dimension of the 720 first historical order sequences of each region is merged into the sample number dimension (the sample number is 1 at this time). The self-attention calculation of the time dimension can be omitted without affecting the parallel calculation of the model. 300 second historical order sequences are obtained, that is, the historical order sequences of 300 regions at the same time point. Through the self-attention neural network model, self-attention calculation is performed between the 300 second historical order sequences. The calculation amount at this time is 300 2 Among them, merging the time dimension of the 720 first historical order sequences in each region into the sample quantity dimension means that the impact of time factors on each region is not considered, but only the mutual influence between different regions is considered.

[0125] S130: Classify multiple first historical order sequences by unit time, and extract time feature vectors from the first historical order sequences in vector form within each unit time period through a self-attention neural network model, including: S131-S132.

[0126] S131: Merge the first historical order sequences in vector form within each unit time period to obtain multiple fourth historical order sequences.

[0127] S132: Perform self-attention calculation between the fourth historical order sequence through the self-attention neural network model to extract the time feature vector.

[0128] Combined with the above example, City A has 300 regions, and there are 300 first historical order sequences in each hour in the past month (720 hours). The spatial dimension of the 300 first historical order sequences in each hour is merged into the sample quantity dimension to obtain 720 fourth historical order sequences, that is, the historical order sequences of 720 hours in the same region. Through the self-attention neural network model, self-attention calculation is performed between the 720 fourth historical order sequences, and the calculation amount is 720 2 Among them, merging the spatial dimension of the 300 first historical order sequences of each hour into the sample quantity dimension means that the influence between regions in each hour is not considered, but only the mutual influence between different hours is considered.

[0129] The total computation of the traditional self-attention neural network model is (300*720) 2 times, the amount of calculation in this embodiment is (300 2 +720 2 ) times. Thus, this embodiment changes the number of calculations of the self-attention neural network model from full to sparse, which improves the model calculation efficiency and also makes the model easier to converge.

[0130] See also Figure 3 As another optional implementation in the embodiment of the present application, the self-attention neural network model includes an encoder, and the encoder includes a first multi-head self-attention layer and a second multi-head self-attention layer. S120: Classify multiple first historical order sequences by region, and extract spatial feature vectors from the historical order sequences in vector form in each region through the self-attention neural network model, including S123-S127.

[0131] S123: Merge the first historical order sequences in the form of vectors in each region to obtain multiple second historical order sequences.

[0132] For example, City A has a total of 300 regions. Since there are 720 hours in the past month, each region has 720 first historical order sequences. The time dimension of the 720 first historical order sequences of each region is merged into the sample quantity dimension to obtain 300 second historical order sequences.

[0133] S124: Grouping the regions based on the closeness of the exchanges between the regions to obtain a plurality of spatial groups.

[0134] The regions within each spatial group are regions with a high degree of traffic between each other. These regions are often the starting point and destination of each other's transportation orders. For example, in the morning, passengers will go from residential area A to business district B, and in the evening, they will return from business district B to residential area A. In this case, the traffic between these two regions is relatively close. In the morning, the destination of the transportation order starting from area A is likely to be area B; then the transportation orders generated from area A in the past hour are likely to arrive at the destination area B in large numbers one hour later. By grouping these regions with a high degree of traffic into the same spatial group, the self-attention neural network model can more easily learn the spatial dependencies between them.

[0135] For example, Considered as the closeness of contacts between regions, the k-means clustering algorithm is used according to To reorganize the regions, the pseudo code could be as follows:

[0136] Input: The set of target urban areas D = { , ,…, };

[0137] The number of groups (clusters) of regions reorganized is k.

[0138] Process: Randomly select k regions from D as initial centers

[0139] repeat

[0140] Order cluster partition = empty set (i=1~k)

[0141] for x=1,2,…,m do

[0142] Query the closeness of the contacts between each region x and each initial central region;

[0143] Determine the cluster label of x according to the center with the highest degree of compactness;

[0144] end for

[0145] for C=1,2,…,k do

[0146] Calculate the new center of each cluster (select the area with the highest average density of contacts as the new center of the cluster);

[0147] end for

[0148] until the central area has not been updated.

[0149] Output: The number of groups (clusters) of regional reorganizations divided into C={ , ,…, }.

[0150] S125: Through the first multi-head self-attention layer, self-attention calculation is performed between the second historical order sequences in each spatial group to extract the local spatial feature vector.

[0151] The local spatial eigenvector can reflect the spatial dependence between regions within each spatial group.

[0152] S126: Merge all second historical order sequences in the space group to obtain multiple third historical order sequences.

[0153] S127: Through the second multi-head self-attention layer, self-attention calculation is performed between each third historical order sequence to extract the global spatial feature vector.

[0154] In addition to calculating the self-attention between regions with a high degree of compactness to extract local spatial dependencies, this embodiment also calculates the self-attention between spatial groups with a low degree of compactness, so that all regions can be covered and global spatial dependencies can be extracted. The global spatial feature vector can reflect the spatial dependencies between all regions of the target city.

[0155] The self-attention neural network model of this embodiment is designed to invest more computing resources in areas with high density. Therefore, for areas with low density, the number of calculations should be controlled to improve the computing efficiency. Therefore, all areas are divided into multiple spatial groups. Self-attention calculations are performed between areas in the spatial group, and limited self-attention calculations are shared between these spatial groups. For example, there are 300 areas in total, and now every 30 areas with high density are 1 group, and there are 10 groups with high density in total. Then each spatial group only needs to calculate 30 areas in the group. 2 times of self-attention, plus 10 between spatial groups 2 Shared self-attention, a total of (30 2 +10 2 ) calculations, we can get the global spatial dependency, without having to calculate the full amount of 300 self-attentions (i.e. 3002 Note: The above calculation amount is calculated from the perspective of 30 regions in a spatial group of the self-attention neural network. If calculated from the perspective of the model as a whole, it should be multiplied by the number of spatial groups, that is (10*30 2 +10 2 ) times self-attention.

[0156] This embodiment reorganizes the areas with high degree of communication, and then performs self-attention calculation between the areas in the spatial group, allowing the self-attention neural network model to learn the local spatial dependencies within the spatial group, and then share the self-attention between the spatial groups to learn the global spatial dependencies, thereby realizing the transition from full self-attention calculation to sparse self-attention calculation in space, greatly improving the computational efficiency of the model and making it easier to converge.

[0157] The encoder also includes a third multi-head self-attention layer, a fourth multi-head self-attention layer, and a fifth multi-head self-attention layer. Figure 4 , S130: classify multiple first historical order sequences by unit time, and extract time feature vectors from the first historical order sequences in vector form within each unit time period through a self-attention neural network model, including S133-S138.

[0158] S133: Merge the first historical order sequences in vector form within each unit time period to obtain multiple fourth historical order sequences.

[0159] For example, City A has 300 regions, and there are 300 first historical order sequences in each hour in the past month (720 hours). The spatial dimension of the 300 first historical order sequences in each hour is merged into the sample number (the sample number is 1 at this time) dimension to obtain 720 fourth historical order sequences.

[0160] S134: Through the third multi-head self-attention layer, self-attention calculation is performed between the fourth historical order sequences within each day to extract the time feature vector of the unit time period.

[0161] For example, in the time dimension, the fourth historical order sequence within 24 hours of a day is processed by 24 2 The self-attention calculation is performed to extract the time feature vector of 24 hours in the day, so the calculation amount is 24 2 times. Thus, the time dependency of 24 hours a day can be obtained. For example, there are more people taking taxis at 8 o'clock in the morning every day, and the traffic jam at 8 o'clock will also affect the number of transportation orders at 9 o'clock. For another example, if the target city is Shenzhen, the morning peak may be from 7 to 9 o'clock. If the target city is Urumqi, the morning peak may be from 8 to 10 o'clock.

[0162] Optionally, environmental information is also embedded in the third multi-head self-attention layer so that the third multi-head self-attention layer also extracts the environmental information feature vector within each unit time period; the environmental information includes at least one of the weather, temperature, wind speed, and road condition information corresponding to each unit time period.

[0163] It is easy to understand that, for example, bad weather such as heavy rain / snowfall usually leads people to be more inclined to use taxi or online car-hailing services, thereby increasing the number of orders. In good weather, people may prefer to walk or ride, resulting in a decrease in orders. In hot weather, especially in summer, people are more likely to choose to take a taxi to avoid the high temperature, which will increase the number of orders. Cold weather may also lead people to choose to take a taxi, especially during peak hours in the morning and evening. Strong winds may affect people's willingness to travel, especially during outdoor activities, which may lead to an increase in orders. Similarly, different road conditions, such as traffic jams, road repairs, traffic accidents, large-scale events, etc., will affect the number of transportation orders. Therefore, by embedding the environmental information of each hour into the third multi-head self-attention layer of the encoder, the encoder can extract the environmental information feature vector, and then consider the impact of environmental information on the number of transportation orders when predicting future transportation orders in the future.

[0164] S135: If the first time period is greater than one week, the fourth historical order sequences of each day are merged to obtain multiple fifth historical order sequences.

[0165] For example, there are 7 days in a week and 24 hours in a day. The 24 fourth historical order sequences of each day are combined to obtain 7 fifth historical order sequences, so the calculation amount is 7 2 times. This allows the extraction of time dependencies within the day of the week.

[0166] S136: Through the fourth multi-head self-attention layer, self-attention calculation is performed between the fifth historical order sequence within each week to extract the time feature vector of the day.

[0167] For example, self-attention calculation is performed between the fifth historical order sequence of 7 days in a week, and a total of 7 2 The time feature vectors of the seven days of the week are extracted by self-attention. Thus, the time dependency of the seven days of the week can be extracted. For example, the number of transportation orders during the morning rush hour from Monday to Friday is greater than that on weekends.

[0168] S137: If the first time period is greater than or equal to one month, the fifth historical order sequences of each week are merged to obtain multiple sixth historical order sequences.

[0169] For example, there are 4 weeks in a month, and the fifth historical order sequences of 7 days a week are merged to obtain 4 sixth historical order sequences in total.

[0170] S138: Through the fifth multi-head self-attention layer, self-attention calculation is performed between the sixth historical order sequences within each month to extract the weekly time feature vector.

[0171] For example, self-attention calculation is performed between the sixth historical order sequence of 4 weeks in a month, and a total of 4 2 Second self-attention. Thus, the time feature vectors of the four weeks of each month can be extracted. For example, the number of transportation orders in the fourth week of each month is more than that in the first three weeks.

[0172] Through the above method, the time-dependent extraction is never grouped (24*30) 2 =720 2 The number of self-attention calculations is reduced to (24 2 +7 2 +4 2 ) times, realizing the transition from full self-attention calculation to sparse self-attention calculation in time, greatly improving the computational efficiency of the model and making it easier for the model to converge. Note: The above calculation amount is calculated from the perspective of 24 hours a day, 7 days a week, and 4 weeks a month in the self-attention neural network. If calculated from the perspective of the entire model, it should be multiplied by the number of days and weeks in a month, that is, (30*24 2 +4*7 2 +4 2 ) times self-attention.

[0173] It is easy to understand that if the first time period is greater than one year, it is also possible to extract monthly and / or quarterly time feature vectors according to the above method.

[0174] After the regions are grouped, the encoder branch network sequentially extracts the local spatial feature vector, the global spatial feature vector, the hourly feature vector, the daily feature vector, the weekly feature vector, and the influence of the environmental information. Figure 2 The broken line arrows and ⊕ in the figure represent residual connections. These feature vectors are extracted in Figure 2 It is represented by the local spatial self-attention calculation within the spatial group (within the cluster), the global spatial self-attention calculation between spatial groups (between clusters), the time self-attention calculation in the hour dimension, the time self-attention calculation in the day and week dimensions, and the self-attention calculation of the environmental information:

[0175] ;

[0176] ;

[0177] ;

[0178] ;

[0179] ;

[0180] ;

[0181] in, Represents the regional reorganization as C={ , ,…, } clusters, the self-attention calculation of the internal region of the cluster (that is, the region inside the spatial group) is represents the self-attention calculation between C clusters (i.e., between spatial groups); , , Represents the self-attention calculation of the fourth historical order sequence, the fifth historical order sequence, and the sixth historical order sequence for hours, days, and weeks. Indicates the fourth historical order sequence in the region and additional factors such as weather for self-attention calculation.

[0182] Exemplarily, in order not to affect the parallel calculation of the self-attention neural network model, this embodiment uses the tensor reorganization technology of the eniops library of the Python language to perform tensor transformation before the self-attention calculation of each multi-head self-attention layer:

[0183] from eniops import rearrange, import the tensor transformation function rearrange of the eniops library.

[0184] rearrange( x, 'b (hdwsn) f' ->'(bhdws) nf)') extracts region n from the spatial group and merges the other dimensions in the brackets into the sample quantity dimension b. This allows local spatial self-attention calculation to be performed only on the feature vectors of the regions within the spatial group.

[0185] rearrange( x, 'b (hdwsn) f' ->'(bhdw ) s (nf)') merges the region n within the spatial group with the vector f, extracts the region s between the spatial groups, merges the other dimensions in the brackets into the sample number dimension b, and only calculates the global spatial self-attention of the feature vector of the region s between the spatial groups.

[0186] rearrange( x,'b (hdwsn) f' ->'(bdwsn) h f') , after the spatial self-attention calculation is completed, the spatiotemporal dimension is transformed, the hour dimension h is extracted, and the other dimensions in the brackets are merged into the sample quantity dimension b, and only the hour time self-attention calculation is performed.

[0187] rearrange( x,'b (hdwsn) f' ->' (bwsn) d (hf)') , merge the hour dimension h with the vector f, extract the time dimension day d, merge the other dimensions in the brackets into the training quantity dimension b, and only calculate the time self-attention of the day.

[0188] rearrange( x,'b (hdwsn) f' ->'(bsn) w (dhf)') , merge the hour dimension h and the day dimension d with the vector f, extract the week dimension w, merge the other dimensions in the brackets into the sample number dimension b, and only calculate the weekly time self-attention.

[0189] Where x represents the one-month historical order sequence of all regions, b represents the batch size (sample quantity dimension), h represents the hour, d represents the day, w represents the week, s represents the space group, n represents the region within the space group, and f represents the high-dimensional embedding vector of the second historical order sequence of a certain region.

[0190] Combining S121-S124 and S131-S133, it can be seen that for the first historical order sequence in the self-attention neural network, the computational complexity of the traditional self-attention neural network model is (300*720) 2 The self-attention neural network model of this embodiment reduces the amount of calculation to (30 2 +10 2 +24 2 +7 2 +4 2 ) times. And unlike the encoder structure of the traditional self-attention neural network model that does not distinguish between spatiotemporal dependencies, this embodiment designs a spatiotemporal branch neural network framework, sets different branch networks from the dimensions of space and time, and uses independent network parameters to learn the temporal and spatial dependencies, which are responsible for extracting temporal dependencies and spatial dependencies respectively. Thereby, the complex spatiotemporal cross-dependencies between regions are simplified and decomposed into spatial and temporal dependencies, the regions are reorganized based on the closeness of the exchanges, and the unit time periods are reorganized based on days and weeks, which further reduces the number of self-attention calculations and improves the model calculation efficiency. This embodiment also embeds factor encoding in the encoder, and the self-attention layer corresponding to the "unit time" of the environmental information calculates the self-attention score together with the data of the unit time dimension to extract the influence of the environmental information. The output of the spatiotemporal branch neural network is fused in the last layer of the encoder (feedforward neural network) to extract the spatiotemporal cross-dependencies.

[0191] As an optional implementation in the embodiment of the present application, S140: predicting the traffic travel orders of at least one region in one or more unit time periods in the future according to the time feature vector and the space feature vector through the self-attention neural network model, including:

[0192] S141: The local spatial feature vector, the global spatial feature vector, the time feature vector of the unit time period, the time feature vector of the day, the time feature vector of the week, and the environmental information feature vector output by the encoder are fused in the last layer (feedforward neural network) of the encoder and then used as the input of the decoder.

[0193] S142: Predicting the traffic travel order prediction value of at least one area in one or more unit time periods through a decoder.

[0194] In this embodiment, the output of the encoder is used as the input of the decoder to predict the transportation orders for one or more hours. It is worth noting that at least one of the following information can be obtained based on the transportation orders here: the number of pick-up orders, the number of drop-off orders, the average cost / total cost, the average travel time per order, the average travel distance per order, and the average number of passengers per order.

[0195] As an optional implementation in the embodiment of the present application, the decoder includes a sixth multi-head self-attention layer (i.e. Figure 2 S142: Predicting the traffic order prediction value of at least one region in one or more unit time periods through the decoder, including:

[0196] S143: Before predicting the traffic travel order prediction value of at least one region in one or more unit time periods by the decoder, the method further includes:

[0197] S144: Obtain a preset number of placeholders.

[0198] For example, the user inputs 4 placeholders, and the 4 placeholders represent the transportation travel orders to be predicted in 300 regions in the next 4 hours.

[0199] S145: Through the sixth multi-head self-attention layer, self-attention calculation is performed between the first historical order sequence in the form of a vector of at least one region in the most recent one or more unit time periods and the placeholder, so that the placeholder obtains the spatiotemporal feature vector of at least one region in the most recent one or more unit time periods.

[0200] For example, from the historical order sequences of the past month, we obtain the historical order sequences of 300 regions in the past 4 hours, a total of 4*300=1200 first historical order sequences. We perform self-attention calculations on these 1200 first historical order sequences and 4*300=1200 randomly initialized order sequences of placeholders, that is, the amount of calculation is (1200+1200) 2 times. Thus, the placeholder obtains the spatiotemporal feature vectors of 300 regions in the past 4 hours.

[0201] S146: Through the seventh multi-head self-attention layer, a cross-self-attention calculation is performed on the spatiotemporal feature vector of the placeholder and the fused feature vector of the seventh multi-head self-attention layer of the input decoder to update the spatiotemporal feature vector of the placeholder.

[0202] Compared with the embodiments of S141-S142, the reason why this embodiment performs the sixth layer of multi-head self-attention calculation is to allow the self-attention neural network model to deepen the memory of the last 4 hours, so that the self-attention neural network model pays more attention to recent data to cope with the impact of recent emergencies on transportation orders, such as traffic accidents, large-scale events, heavy rains, strong winds and other factors on transportation orders.

[0203] Correspondingly, S142: predicting the traffic travel order prediction value of at least one region in one or more unit time periods through a decoder, including:

[0204] S147: Outputting, through a decoder, a predicted order sequence of at least one region in a preset number of unit time periods in placeholders corresponding to a preset number of placeholders.

[0205] S148: Obtain predicted transportation travel orders based on the predicted order sequence.

[0206] The predicted order sequence includes at least one transportation order, so the predicted transportation order can be obtained according to the predicted order sequence.

[0207] This embodiment adopts an encoder-decoder network. In the decoder part, a one-time multi-step prediction decoder network framework is proposed. The output of the encoder is used as the input of the one-time multi-step prediction decoder network. First, T placeholders are randomly initialized. , represents the number of future travel orders to be predicted in T steps, and then merges the first historical order sequence in the form of a vector of real data from the recent few hours ,get As input to the decoder:

[0208] ;

[0209] in Represents the high-level feature dimension of the input value, are the length of the first historical order sequence of the last 4 hours input by the decoder and the length of the predicted order sequence of the 4-hour placeholder output by the decoder, Represents the feature vector. When calculating the self-attention score, the first historical order sequence in vector form Self-attention calculation is performed internally, and self-attention calculation is performed with each placeholder, but self-attention calculation is not performed between placeholders. In order to achieve this without affecting the parallel calculation of the model, this embodiment proposes a novel mask matrix to mask the self-attention score, such as Figure 2 As shown, the principle is to replace the self-attention scores between placeholders with 0. In this way, only one model forward propagation is needed to directly predict the number of travel demand orders in each region of the city in the next T steps, without the need for stepwise autoregressive prediction.

[0210] Optional. The encoder and decoder of the self-attention neural network model of the embodiment of the present application each have 6 layers, thereby improving the model's expressiveness, information processing capability, and training efficiency, so that the model can better process complex and large amounts of historical order sequence data.

[0211] The decoder network framework of the one-time multi-step prediction in this embodiment is different from the traditional recursive step-by-step prediction. The decoder network designed in this embodiment inputs a randomly generated placeholder with the required number of prediction steps (a few hours in the future), calculates the self-attention of the placeholder and the decoder network output at one time, and masks the self-attention scores with other placeholders. It can quickly output multi-step prediction values ​​at one time, thereby improving the efficiency of model reasoning and prediction.

[0212] The embodiment of the present application realizes self-attention calculation only for local areas by deeply combining clustering and spatial dimension tensor transformation technology in the self-attention neural network model. ，And the shared self-attention calculation for the global region, extracting the spatial feature vector from the first historical order sequence in multiple regions; then, by deeply combining the tensor transformation technology of the time dimension in the self-attention neural network model, extracting the short-term, medium-term, and long-term time feature vectors from the historical order sequence in the unit time period of each hour, day, and week; finally, through the placeholder and special mask matrix design, the self-attention neural network model predicts the transportation travel orders of at least one region in one or more unit time periods in the future at one time (without autoregression) based on the time feature vector and the spatial feature vector. Therefore, the embodiment of the present application decomposes the complex spatiotemporal features, and gradually extracts the local and global spatial feature vectors and the time feature vectors of hours, days, and weeks from the historical order sequence in different self-attention neural network layers, and then combines the two to predict future transportation travel orders, thereby changing the number of calculations of the model from full to sparse, improving the calculation efficiency of the model.

[0213] See also Figure 5 , Figure 5 This is a visualized heat map of the traffic orders in the target city in the next 3 hours predicted by the above-mentioned traffic order prediction method. It can be seen from the figure that the predicted value and the true value are very close, indicating that the embodiment of the present application can have a higher prediction accuracy while greatly reducing the amount of calculation.

[0214] The above embodiments are the deployment stage of the self-attention neural network model, but before deployment, a model training stage is required. That is, the self-attention neural network model is obtained by training the initial self-attention neural network model through the historical order sequence. The model training stage can be implemented in the following way:

[0215] Obtain the hourly transportation orders of 300 areas in the target city in the past year, and obtain 365*24*300 first historical order sequences.

[0216] The first historical order sequence is divided into a training set, a validation set, and a test set. In the model training phase, the order sequence for one or more hours in the future is used as the target prediction value. Using the 30*24*300 first historical order sequences of the past month as a training sample, the model outputs the order sequence for one or more hours in the future. Similarly, in order to obtain more training samples, the time slides back one or more hours, and the first historical order sequence of the past month is used again as a new training sample to predict the order sequence for one or more hours in the future. During model training, the predicted values ​​and true values ​​obtained from a batch (for example: 32) of training samples are used to adjust the model parameters.

[0217] The initial self-attention neural network model is trained by the training set. The training method can be the same as the S120-S140 method, so that the predicted traffic travel order can be obtained, and the pre-test is input into the loss function, and the loss function is converged through continuous iteration. During the training process, the loss function value of each round is recorded to facilitate the observation of the training progress of the initial self-attention neural network model. In addition, indicators such as the square error on the training set and the test set are calculated to evaluate the performance of the initial self-attention neural network model. After the training is completed, the self-attention neural network model is obtained.

[0218] Among them, the loss function is as follows:

[0219] ,

[0220] in, is the root mean square difference between the predicted value and the true value of all regions in the next few hours. I means that the target city is divided into I regions. T means that the model predicts the order demand in the next T steps (for example, the next T hours). represents the actual travel order demand of the i-th area in the target city in the next t hours, including the number of pick-up orders and drop-off orders. is an estimate of the model output.

[0221] After the training is completed, a self-attention neural network model is obtained, which can be used in transportation order prediction platforms (such as taxis, online car-hailing, and shared bicycle platforms) and provide services in the form of http interfaces.

[0222] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0223] Corresponding to the transportation order prediction method described in the above embodiment, Figure 6 A structural block diagram of a transportation order prediction device provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0224] Reference Figure 6 , the device comprises:

[0225] The acquisition module 210 is used to acquire the transportation travel orders in each unit time period of multiple areas in the target city within the first time period to obtain multiple first historical order sequences.

[0226] The spatial feature extraction module 220 is used to classify multiple first historical order sequences by region, and extract spatial feature vectors from the first historical order sequences in vector form within multiple regions through a self-attention neural network model.

[0227] The time feature extraction module 230 is used to classify multiple first historical order sequences in unit time, and extract the time feature vector from the first historical order sequence in vector form within each unit time period through a self-attention neural network model.

[0228] The prediction module 240 is used to predict the traffic travel orders of at least one region in one or more unit time periods in the future according to the time feature vector and the space feature vector through a self-attention neural network model.

[0229] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0230] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0231] An embodiment of the present application also provides an electronic device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor implements the steps of any of the above-mentioned method embodiments when executing the computer program.

[0232] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0233] An embodiment of the present application provides a computer program product. When the computer program product runs on a mobile terminal, the mobile terminal can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0234] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to an electronic device, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0235] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0236] The computer program code for performing the operation of the embodiment of the application can be written with one or more programming languages ​​or their combination, and the programming language includes object-oriented programming languages, such as python, Java, Smalltalk, C++, and also includes conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed completely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on the remote computer, or executed completely on the remote computer or server. In the case of a remote computer, the remote computer can include a local area network (LAN) or a wide area network (WAN)--connected to the user's computer through any type of network, or, can be connected to an external computer (for example, utilizing an Internet service provider to connect through the Internet).

[0237] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0238] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0239] In the embodiments provided in the present application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0240] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0241] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for predicting transportation orders, characterized in that: include: Obtaining the transportation orders of multiple areas of the target city in each unit time period during the first time period, and obtaining multiple first historical order sequences; Classifying the plurality of first historical order sequences by the region, and extracting spatial feature vectors from the plurality of first historical order sequences in vector form within the region by using a self-attention neural network model; Classifying the plurality of first historical order sequences by the unit time, and extracting a time feature vector from the first historical order sequence in vector form within each of the unit time periods by using the self-attention neural network model; Predicting the transportation travel orders of at least one region in one or more future unit time periods according to the time feature vector and the space feature vector by using the self-attention neural network model; The self-attention neural network model includes an encoder, wherein the encoder includes a first multi-head self-attention layer and a second multi-head self-attention layer; The plurality of first historical order sequences are classified according to the region, and spatial feature vectors are extracted from the first historical order sequences in vector form in the plurality of regions through a self-attention neural network model, including: Merging the first historical order sequences in the form of vectors in each of the regions to obtain a plurality of second historical order sequences; The regions are grouped based on the density of travel between the regions to obtain a plurality of spatial groups; wherein the density of travel between the regions is determined according to at least one of the order quantity, travel time, and travel distance of the transportation travel orders per unit time; Through the first multi-head self-attention layer, self-attention calculation is performed between the second historical order sequences in each of the spatial groups to extract local spatial feature vectors; Merging all second historical order sequences in the space group to obtain multiple third historical order sequences; Through the second multi-head self-attention layer, self-attention calculation is performed between each of the third historical order sequences to extract a global spatial feature vector; The encoder also includes a third multi-head self-attention layer, a fourth multi-head self-attention layer, and a fifth multi-head self-attention layer; Classifying the plurality of first historical order sequences by the unit time, and extracting a time feature vector from the first historical order sequence in vector form within each unit time period by the self-attention neural network model, comprises: Merging the first historical order sequences in vector form within each of the unit time periods to obtain a plurality of fourth historical order sequences; Through the third multi-head self-attention layer, self-attention calculation is performed between the fourth historical order sequences within each day to extract the time feature vector of the unit time period; If the first time period is greater than one week, merging the fourth historical order sequences of each day to obtain multiple fifth historical order sequences; Through the fourth multi-head self-attention layer, self-attention calculation is performed between the fifth historical order sequences within each week to extract the time feature vector of the day; If the first time period is greater than or equal to one month, merging the fifth historical order sequences of each week to obtain multiple sixth historical order sequences; Through the fifth multi-head self-attention layer, self-attention calculation is performed between the sixth historical order sequences within each month to extract the weekly time feature vector; Environmental information is also embedded in the third multi-head self-attention layer, so that the third multi-head self-attention layer also extracts the environmental information feature vector of each unit time period; the environmental information includes at least one of the weather, temperature, wind speed, and road condition information corresponding to each unit time period.

2. The method for predicting transportation orders according to claim 1, characterized in that: The predicting, by the self-attention neural network model according to the time feature vector and the space feature vector, the transportation travel orders of at least one region in one or more future unit time periods includes: The local space feature vector, the global space feature vector, the time feature vector of a unit time period, the time feature vector of a day, the time feature vector of a week, and the environmental information feature vector output by the encoder are fused as the input of the decoder; The decoder is used to predict the transportation travel order forecast value of at least one of the regions in one or more of the unit time periods.

3. The method for predicting transportation orders according to claim 2, characterized in that: The decoder comprises a sixth multi-head self-attention layer and a seventh multi-head self-attention layer; Before predicting the predicted value of the transportation order of at least one of the regions in one or more of the unit time periods by the decoder, the method further includes: Get a preset number of placeholders; By means of the sixth multi-head self-attention layer, a self-attention calculation is performed between the first historical order sequence in the form of a vector of at least one of the regions in the most recent one or more unit time periods and the placeholder, so that the placeholder obtains the spatiotemporal feature vector of at least one of the regions in the most recent one or more unit time periods; Performing a cross-self-attention calculation on the spatiotemporal feature vector of the placeholder and the fused feature vector of the seventh multi-head self-attention layer input to the decoder through the seventh multi-head self-attention layer to update the spatiotemporal feature vector of the placeholder; The predicting, by the decoder, the predicted value of the transportation order of at least one of the regions in one or more of the unit time periods comprises: Outputting, by the decoder, a predicted order sequence of at least one of the regions in the preset number of the unit time periods in the placeholders corresponding to the preset number of the placeholders; According to the predicted order sequence, the predicted transportation orders are obtained.

4. The method for predicting transportation orders according to claim 1, characterized in that: The classifying the plurality of first historical order sequences by the region, and extracting spatial feature vectors from the first historical order sequences in vector form within the plurality of regions through a self-attention neural network model, comprises: Merging the first historical order sequences in the form of vectors in each of the regions to obtain a plurality of second historical order sequences; By using the self-attention neural network model, performing self-attention calculation between the second historical order sequences to extract the spatial feature vector; Classifying the plurality of first historical order sequences by the unit time, and extracting a time feature vector from the first historical order sequence in vector form within each unit time period by using the self-attention neural network model, including: Merging the first historical order sequences in vector form within each of the unit time periods to obtain a plurality of fourth historical order sequences; Through the self-attention neural network model, self-attention calculation is performed between the fourth historical order sequence to extract the time feature vector.

5. The method for predicting transportation orders according to any one of claims 1 to 4, characterized in that: After obtaining the plurality of first historical order sequences, the method further includes: Preprocessing the plurality of first historical order sequences; According to the pre-processed plurality of first historical order sequences, a topological graph network corresponding to each unit time is constructed; wherein each region represents a node in the topological graph network, and if two regions are the starting point and destination of the transportation orders for each other, and the number of transportation orders in one or more unit time periods is greater than a preset threshold, a connection is established between the two regions, and the weight of the connection is determined according to at least one of the number of transportation orders, travel time, and travel distance in a unit time; and the weight of the connection is used as the closeness of the exchanges between the regions; The topological graph network is converted into a vector form to obtain the first historical order sequence in a vector form.

6. A transportation order prediction device, characterized in that: include: An acquisition module is used to acquire the transportation travel orders of multiple areas of the target city in each unit time period during the first time period to obtain multiple first historical order sequences; a spatial feature extraction module, configured to classify the plurality of first historical order sequences by the region, and extract spatial feature vectors from the plurality of first historical order sequences in vector form within the region by using a self-attention neural network model; the self-attention neural network model comprises an encoder, and the encoder comprises a first multi-head self-attention layer and a second multi-head self-attention layer; A time feature extraction module, used to classify the plurality of first historical order sequences according to the unit time, and extract a time feature vector from the first historical order sequence in vector form within each of the unit time periods through the self-attention neural network model; A prediction module, configured to predict the transportation travel orders of at least one region in one or more future unit time periods according to the time feature vector and the space feature vector through the self-attention neural network model; The time feature extraction module comprises: Merging the first historical order sequences in the form of vectors in each of the regions to obtain a plurality of second historical order sequences; The regions are grouped based on the density of travel between the regions to obtain a plurality of spatial groups; wherein the density of travel between the regions is determined according to at least one of the order quantity, travel time, and travel distance of the transportation travel orders per unit time; Through the first multi-head self-attention layer, self-attention calculation is performed between the second historical order sequences in each of the spatial groups to extract local spatial feature vectors; Merging all second historical order sequences in the space group to obtain multiple third historical order sequences; Through the second multi-head self-attention layer, self-attention calculation is performed between each of the third historical order sequences to extract a global spatial feature vector; The encoder also includes a third multi-head self-attention layer, a fourth multi-head self-attention layer, and a fifth multi-head self-attention layer; Classifying the plurality of first historical order sequences by the unit time, and extracting a time feature vector from the first historical order sequence in vector form within each unit time period by the self-attention neural network model, comprises: Merging the first historical order sequences in vector form within each of the unit time periods to obtain a plurality of fourth historical order sequences; Through the third multi-head self-attention layer, self-attention calculation is performed between the fourth historical order sequences within each day to extract the time feature vector of the unit time period; If the first time period is greater than one week, merging the fourth historical order sequences of each day to obtain multiple fifth historical order sequences; Through the fourth multi-head self-attention layer, self-attention calculation is performed between the fifth historical order sequences within each week to extract the time feature vector of the day; If the first time period is greater than or equal to one month, merging the fifth historical order sequences of each week to obtain multiple sixth historical order sequences; Through the fifth multi-head self-attention layer, self-attention calculation is performed between the sixth historical order sequences within each month to extract the weekly time feature vector; Environmental information is also embedded in the third multi-head self-attention layer, so that the third multi-head self-attention layer also extracts the environmental information feature vector of each unit time period; the environmental information includes at least one of the weather, temperature, wind speed, and road condition information corresponding to each unit time period.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 5 is implemented.

Images