Intercity public and private vehicle multi-modal travel path multi-agent collaborative reasoning restoration method and system

Abstract

The application discloses an intercity public and private vehicle multi-mode travel path multi-agent collaborative reasoning restoration method and system, and realizes reasoning restoration of a real travel path of a traveler based on sparse trajectory observation data of intercity travel of the traveler. The steps are as follows: original trajectory observation data and traffic network data are acquired and preprocessed; the road network data and the trajectory observation data are subjected to gridding treatment and storage; a path search agent is designed and trained to realize reasoning of reasonable restoration of a path sequence; a mode recognition agent is designed and trained to realize reasoning of a reasonable travel mode sequence; and the two trained agents are collaboratively reasoned to realize restoration of the real path of intercity public and private vehicle multi-mode travel. The application decouples the complex intercity public and private vehicle multi-mode travel path restoration problem into two sub-problems of path search and mode recognition, constructs two agents to respectively solve the two sub-problems, realizes reasoning restoration of the real path of intercity travel based on sparse observation trajectories, and provides a new idea and method for the path restoration field.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence and intelligent transportation, in particular to an intercity public and private multi-modal travel path multi-agent collaborative reasoning restoration method and system. BACKGROUND

[0002] With the rapid development of intelligent transportation systems and the construction of modern transportation networks in China, comprehensive transportation networks are becoming increasingly sophisticated. The railway network, highway network, national and provincial trunk network, and low-grade roads together form a multi-level intercity travel network structure. Residents' intercity travel paths have gradually increased, and travel modes have become increasingly diversified. In this context, a deep understanding of residents' intercity travel patterns is of great significance for improving the efficiency of transportation resource allocation. However, existing residents' intercity travel trajectory observation data often has problems such as sparse spatio-temporal sampling granularity and limited spatial accuracy, which cannot support accurate identification of residents' travel modes and travel paths, posing great challenges to in-depth analysis of residents' intercity travel patterns.

[0003] To date, many scholars have conducted research on path reasoning restoration, aiming to recover residents' true travel paths from sparse and biased trajectory observation data. Traditional path restoration methods based on hidden Markov models rely on the Markov assumption and model the path restoration process as a double stochastic process to recover the path. However, the computational complexity of this method increases exponentially with the length of the travel path, and its path restoration ability is significantly limited in the face of current residents' intercity travel characteristics such as long travel, wide spatial range, and diverse feasible paths. Supervised learning methods are another mainstream path restoration method. This type of method decouples multi-dimensional features from labeled data and directly learns the high-dimensional mapping relationship between observed trajectories and true paths to achieve path restoration. However, the model learning effect of this method is highly dependent on the quality and quantity of labeled data, and the model generalization and transferability are also affected by the distribution of labeled data. Semi-supervised learning and unsupervised learning path restoration methods have lower requirements for labeled data, but relevant research is still relatively rare, and there is a lack of corresponding theoretical support.

[0004] Considering the limitations of existing path restoration methods, the present application aims to explore a new path restoration method that can adapt to the current characteristics of residents' intercity travel, reduce costs as much as possible while maintaining high path restoration accuracy, and provide efficient technical support for intelligent transportation systems.

[0005] The differences between the existing technology are as follows:

[0006] Comparison with the technology of patent CN118966349A "Automatic driving multi-agent future behavior topology reasoning method, device, equipment, medium and product"

[0007] The patent CN118966349A proposes an automatic driving multi-agent future behavior topology reasoning method, device, equipment, medium and product, belonging to the technical field of automatic driving. The patent proposes an intercity public and private multi-modal travel path multi-agent collaborative reasoning restoration method and system, belonging to the field of artificial intelligence and intelligent transportation. The technical fields involved in the two have essential differences.

[0008] The patent CN118966349A uses a collaborative learning model to complete the prediction of the future behavior of automatic driving multi-agents based on a preset fitness function. The patent constructs a multi-agent model and completes the reasoning and restoration of the real travel path based on the output result feedback learning of the agent. The methods used in the two have essential differences.

[0009] The patent CN118966349A constructs an automatic driving multi-agent future behavior topology reasoning method, device, equipment, medium and product, focusing on solving the uncertainty and heterogeneous interaction problems of multi-agent driving scenarios. The patent proposes an intercity public and private multi-modal travel path multi-agent collaborative reasoning restoration method and system, mainly solving the path restoration problem under the sparse observation data of intercity travel trajectories. The purpose and function of the two have essential differences.

[0010] Comparison with the patent CN102904343A "State monitoring system and method based on distributed multi-agent"

[0011] The patent CN102904343A proposes a state monitoring system and method based on distributed multi-agent, belonging to the technical field of substation state detection. The patent proposes an intercity public and private multi-modal travel path multi-agent collaborative reasoning restoration method and system, belonging to the field of artificial intelligence and intelligent transportation. The technical fields involved in the two have essential differences.

[0012] The patent CN102904343A uses multiple same state detection agents to achieve collaborative detection of substation state based on a multi-agent network. The patent constructs path search and mode recognition agents and completes the reasoning and restoration of the real travel path based on the output result feedback learning of each other. The methods used in the two have essential differences.

[0013] The patent CN102904343A proposes a state monitoring system and method based on distributed multi-agent, mainly used to solve the defects of islanded state monitoring and facilitate the organic integration of online monitoring of the entire power grid. The patent proposes an intercity public and private multi-modal travel path multi-agent collaborative reasoning restoration method and system, mainly solving the path restoration problem under the sparse observation data of intercity travel trajectories. The purpose and function of the two have essential differences. SUMMARY

[0014] In the face of the current residents' intercity travel, the travel is long, involves a wide range of space, and has many feasible paths, and the present application proposes an intercity public and private multi-mode travel path multi-agent collaborative reasoning restoration method and system, which realizes the reasoning and restoration of the real path of the intercity travel of the traveler through the learning and training of multiple agents, maintains a high path restoration accuracy, and reduces the dependence on high-quality labeled samples.

[0015] To achieve the above object, the technical scheme adopted by the present application is:

[0016] The intercity public and private multi-mode travel path multi-agent collaborative reasoning restoration method comprises the following steps:

[0017] Step S10, obtaining and preprocessing original trajectory observation data and traffic network data;

[0018] Step S20, performing grid processing and storage on the road network data and the trajectory observation data;

[0019] Step S30, designing and training a path search agent to infer a reasonable restoration path sequence after a given trajectory observation sequence and a specific traffic mode corresponding to the road network;

[0020] Step S40, designing and training a mode recognition agent to infer a reasonable travel mode sequence after a given trajectory observation sequence and a path search agent feedback road matching degree;

[0021] Step S50, path search agent and mode recognition agent asynchronous training, collaborative reasoning, and reinforcement learning training to achieve the reasoning and restoration of the real path corresponding to the sparse trajectory observation sequence.

[0022] As a further improvement of the present application, the step S10 specifically comprises:

[0023] Step S11, obtaining and preprocessing trajectory observation data;

[0024] Obtain trajectory observation data containing intercity travel information of travelers through a specific software or mobile operator, and the obtained data should at least include timestamp information and latitude and longitude information, then preprocess the obtained original trajectory observation data of the intercity travel of the residents, including data filling and repeated removal operation;

[0025] Step S12, obtaining and preprocessing road network data;

[0026] Obtain the traffic network data of the research area, which should at least include the location information of each road, divide the overall road network data into multiple subsets based on the type of the road, respectively corresponding to different traffic modes, including complete road network data of the corresponding traffic mode, and select the road data subset involved in the intercity travel from it for use.

[0027] As a further improvement of the present invention, step S20 specifically includes:

[0028] Step S21, rasterize the research area;

[0029] The study area was gridded using ArcGIS software with appropriate dimensions, and the grid was formatted as (locx) based on its row and column positions. i ,locy i Numbering is done in the form of ) where locx i The column number of the grid, locy i The row number of the grid;

[0030] Step S22: Rasterize and store the road network data;

[0031] Using ArcGIS software, road networks for various modes of transportation are mapped to numbered grids. If a grid contains a road corresponding to a certain mode of transportation, the corresponding value is 1; otherwise, it is 0. After all this is done, the traffic network information for each grid is a string of numbers with a length equal to the number of modes of transportation and a value of 1 or 0. To reduce storage space, this string of 1 / 0 numbers is converted into a decimal number. At this point, the traffic network information for each grid is completely stored. With the traffic network information for each grid complete, the traffic network information for the eight surrounding grids is retrieved and stored. In other words, each grid ultimately stores the traffic network information for nine grids, including itself. The grid order is as follows: top left, top right, top right, left, itself, right, bottom left, bottom right, bottom right.

[0032] Step S23: Rasterize and store trajectory observation data;

[0033] The location information of the current trajectory observation data is still represented by latitude and longitude. By using ArcGIS software, all the location information in the trajectory observation data is mapped to a grid. The storage is changed from latitude and longitude to storage by the row and column of the grid. After the conversion, the trajectory observation data includes the ID, timestamp, column and row of each trajectory observation point.

[0034] As a further improvement of the present invention, step S30 specifically includes:

[0035] Step S31: Design a path search agent;

[0036] At time step t in the path reconstruction process, the path reasoning agent performs a Markov decision process based on the policy, starting from the current state s. t Take action a t Transition to the next state and receive the reward r for this moment.T (s t ,a t ), wherein the state s t obtained by the agent contains road network information corresponding to a given travel mode, position information of the agent at the moment, and position information of the target trajectory point; the action a t taken by the agent is one of the action space, and the action space includes moving to the upper left, the upper, the upper right, the left, the right, the lower left, the lower, and the lower right; the reward r T (s t ,a t ) obtained by the agent is a weighted sum of the following road following reward and end point approaching reward:

[0037] Step S32, training the path searching agent;

[0038] The path searching agent is trained by using a policy-based SAC algorithm, and in the training process, the policy is modeled as π θ by a neural network, θ is a parameter set of the neural network, and then the policy parameter θ is learned by using a gradient descent method to obtain an optimal policy, and the optimization objective of θ is as follows:

[0039]

[0040] Since the single policy network parameter may not be able to adapt to the state difference between different trajectory points, the target-oriented reinforcement learning is used to strengthen the generalization ability of the agent to the diverse state space, the position information about the agent and the target trajectory point in the agent state is mapped into relative position information from the coordinate origin, so as to convert the path reasoning problem between different trajectory points into a series of similar tasks of reaching different target points from the same trajectory point;

[0041] In the training process, an experience replay pool is also used to improve the training efficiency of the agent: in each sampling phase, the state, action, reward, and next state of the current time step of the path reasoning agent are stored in the experience replay pool in the form of a tuple <s, a, r, s'>; in the gradient update phase, a plurality of data are randomly sampled from the experience replay pool to update the gradient of the policy network, which greatly improves the data utilization efficiency.

[0042] As a further improvement of the application, the step S40 specifically comprises:

[0043] Step S41, designing a mode recognition agent;

[0044] At the t-th time step in the mode recognition process, the mode recognition agent performs a Markov decision process according to the policy from the current state s t to take an action at Transfers to the next state and obtains the reward r M (s t ,a t ) at this moment. Among them, the state s t obtained by the agent contains the local dynamic and static features of the observation sequence at this moment, the road network information of the position of the agent, and the road network information of the grid around the position of the agent; the action a t taken by the agent is the identified traffic mode; the reward r M (s t ,a t ) obtained by the agent is set as the product of the road matching degree fed back by the path search agent and the reward obtained according to the motion feature recognition mode;

[0045] Step S42, training the mode recognition agent;

[0046] Unlike the policy-based reinforcement learning process of the path reasoning agent, the mode recognition agent adopts a value-based DQN algorithm for training, and realizes the optimal selection of traffic mode label actions in different states by learning the state-action value function Q(s, a). In each learning and training process, the update rule of the state-action value function Q(s, a) of the agent is:

[0047] Q(s t ,a t )←Q(s t ,a t )+α[r M +γmax a’ Q(s t+1 ,a’)-Q(s t ,a t )]

[0048] In order to effectively find the optimal Q function, a neural network is used to model it; in order to analyze the difference of different actions taken in the same state, the state-action value function Q is expressed as the sum of the state value function V and the advantage function A, so that the agent can better handle the action selection in similar states. The following formula is the expression of the state-action value function Q, wherein V(s t ) is the expected return that the agent can obtain in the state s t , A(s t ,a t ) is the advantage function of the agent taking different actions a t in the state s t , and ω is the parameter set of the neural network;

[0049] Q ω (s t ,a t )=V(st )+A(s t ,a t )

[0050] To obtain the deep connection of the trajectory points and the adjacent trajectory points, the state-action value neural network Q of the mode recognition agent is recognized ω (s t ,a t ) adopts a long short-term memory network layer to replace a full connection layer, and in addition, a target network is introduced in the training process The double neural network is used for training, and the training network Q ω (s,a) is updated every step in the training, and the target network uses historical parameters, and is synchronized with the training network once every C steps, so that the influence of the overestimated network value on the mode recognition performance is avoided.

[0051] The intercity public and private multi-modal travel path multi-agent collaborative reasoning restoration system provided by the application comprises an original data acquisition and preprocessing module, a data grid and storage module, a path search agent design and training module, a mode recognition agent design and training module and a result output module, wherein:

[0052] The original data acquisition and preprocessing module is used for acquiring and preprocessing intercity travel trajectory observation data and road network data, so as to provide a data source for subsequent gridding processing;

[0053] The data grid and storage module is used for gridding and storing the road network data corresponding to different traffic modes and the observation trajectory data implying real paths, so as to facilitate the subsequent path search restoration;

[0054] The path search agent design and training module is used for reasoning out a reasonable restoration path sequence based on a given trajectory observation sequence and a specific traffic mode corresponding road network, so as to calculate a road matching degree as gain information to support the training of the mode recognition agent;

[0055] The mode recognition agent design and training module is used for reasoning out a reasonable travel mode sequence based on a given trajectory observation sequence and the road matching degree fed back by the path search agent, so as to provide correct road network information for the path search agent and support the reasoning and restoration of the real path corresponding to the observation trajectory by the path search agent;

[0056] The result output module is used for completing the reasoning and restoration of the real path and the result output of the path search agent and the mode recognition agent together after a given trajectory observation sequence.

[0057] Beneficial effects:

[0058] The intercity public and private vehicle multi-mode travel path multi-agent collaborative reasoning restoration method and system provided by the present application decouples the complex intercity public and private vehicle multi-mode travel path restoration problem into two sub-problems of path search and mode identification, and solves the sub-problems by constructing two reinforcement learning agents, thereby realizing efficient reasoning and restoration of the real path of the intercity traveler while reducing the cost. BRIEF DESCRIPTION OF DRAWINGS

[0059] Figure 1 is a flowchart of the present application;

[0060] Figure 2 is an architecture diagram of the original data acquisition and preprocessing of the present application;

[0061] Figure 3 is an architecture diagram of the data gridding and storage of the present application;

[0062] Figure 4 is an architecture diagram of the collaborative training of the path search agent and the mode identification agent of the present application;

[0063] Figure 5 is a grid division and road network display of the research area in the embodiment of the present application;

[0064] Figure 6 is a trajectory observation data gridding processing result display in the embodiment of the present application;

[0065] Figure 7 is a training process display of the path search agent in the embodiment of the present application;

[0066] Figure 8 is a training process display of the mode identification agent in the embodiment of the present application;

[0067] Figure 9 is a path restoration result display output in the embodiment of the present application. DETAILED DESCRIPTION

[0068] The present application will be further described in detail below in combination with the drawings and specific embodiments:

[0069] The present application is widely applicable. In the case that the administrative division information and road network information of the research area are complete, the real travel path can be reasoned and restored based on the sparse trajectory observation data of the intercity travel of the residents in the area. In the following embodiment, Jiangsu Province is taken as the research area, and the real path of the intercity travel of the residents in the area based on high-speed rail, common rail, expressway and national and provincial highway is reasoned and restored.

[0070] EMBODIMENT

[0071] As Figure 1As shown, the intercity public and private multi-modal travel path multi-agent collaborative reasoning restoration method and system of the embodiment includes the following steps:

[0072] Step S10, obtaining and preprocessing original trajectory observation data and traffic network data;

[0073] Step S20, performing grid processing and storage on the road network data and trajectory observation data;

[0074] Step S30, designing and training path search agents to infer reasonable restoration path sequences after given trajectory observation sequences and specific traffic mode corresponding road networks;

[0075] Step S40, designing and training mode recognition agents to infer reasonable travel mode sequences after given trajectory observation sequences and path search agent feedback road matching degrees;

[0076] Step S50, path search agents and mode recognition agents are trained asynchronously and collaboratively, and after reinforcement learning training is completed, they jointly realize the reasoning restoration of sparse trajectory observation sequences corresponding to real paths.

[0077] As shown, Figure 2 the step S10 specifically includes:

[0078] Step S11, obtaining and preprocessing trajectory observation data;

[0079] Through the mobile operator "China Unicom", trajectory observation data containing intercity traveler travel information in Jiangsu Province, mobile signaling data, containing time stamps reflecting time information and latitude and longitude reflecting position information, are obtained. However, due to the large time and space sampling interval of mobile signaling data, or the signal interference and other abnormal conditions of data collection equipment, mobile signaling data often presents sparseness, and there are problems such as data missing, a large amount of repeated data, etc. Therefore, the obtained mobile signaling data is preprocessed, including data filling, repeated data removal, etc. to improve the availability and consistency of the data;

[0080] Step S12, obtaining and preprocessing road network data;

[0081] The traffic network data of Jiangsu Province is obtained from the OpenStreetMap website or other open source repositories, which should at least include the location information of each road. However, the original traffic network data is a collection of all road data in the study area, which is large in size and contains redundant data. Therefore, based on the type of the road, the overall road network data is divided into multiple subsets, each corresponding to a different mode of transportation, including complete road network data for the corresponding mode of transportation, and the subsets of road data related to intercity travel such as high-speed rail, ordinary rail, expressway, national and provincial roads are selected for use.

[0082] As shown in Figure 3 , the step S20 specifically includes:

[0083] Step S21, rasterizing the study area;

[0084] The Jiangsu Province area is divided into grids by ArcGIS software. After division, the overall grid specification is 529 rows x 564 columns, and the unit grid length is 1000 meters in both horizontal and vertical directions. The grid is numbered in the form of (locx i , locy i ), where locx i is the column number of the grid, and locy i is the row number of the grid. The grid division and road network display of Jiangsu Province are shown in Figure 5 ;

[0085] Step S22, rasterizing road network data and storing;

[0086] Each mode of transportation is mapped to the numbered grid by ArcGIS software. If the grid has a road corresponding to a certain mode of transportation, the corresponding value is 1, otherwise it is 0. After completion, the traffic network information of each grid itself is a string of numbers with a length of the number of transportation modes, with values of 1 or 0. In order to reduce storage space, the 1 / 0 number string is converted to a decimal number. At this point, the traffic network information of each grid itself is completely stored. In the case where each grid has complete traffic network information corresponding to itself, the traffic network information of the surrounding eight grids is retrieved and stored, i.e. each grid finally stores the traffic network information of nine grids including itself, and the grid order is as follows: upper left, upper, upper right, left, itself, right, lower left, lower, and lower right;

[0087] Step S23, trajectory observation data rasterization and storage;

[0088] Currently, the location information of the trajectory observation data is still represented as latitude and longitude. To facilitate path searching and reconstruction, ArcGIS software is used to map all location information in the trajectory observation data to a grid. The storage method is changed from latitude and longitude to storage based on the row and column of the corresponding grid. After conversion, the trajectory observation data includes the ID, timestamp, column, and row of each trajectory observation point. See [link to previous image / image ... Figure 6 .

[0089] like Figure 4 As shown, step S30 specifically includes:

[0090] Step S31: Design a path search agent;

[0091] At time step t in the path reconstruction process, the path reasoning agent performs a Markov decision process based on the policy, starting from the current state s. t Take action a t Transition to the next state and receive the reward r for this moment. T (s t ,a t (wherein, the state s acquired by the agent). t This includes road network information corresponding to the given mode of transportation, the agent's current location, and the location information of the target trajectory point; the action 'a' taken by the agent. t This is one of the action spaces, which includes movement to the upper left, directly above, upper right, directly left, directly right, lower left, directly below, and lower right; the reward r obtained by the agent. T (s t ,a t This is a weighted sum of the follow-the-road bonus and the near-finish bonus;

[0092] Step S32: Train the path search agent;

[0093] The path-search agent is trained using the policy-based SAC algorithm. During training, the policy is modeled as π by the neural network. θ Let θ be the set of parameters of the neural network. Then, the policy parameters θ are learned using the gradient descent method to obtain the optimal policy. The optimization objective of θ is as follows:

[0094]

[0095] Since a single policy network parameter may not be able to adapt to the state differences between different trajectory points, goal-oriented reinforcement learning is used to enhance the agent's ability to generalize to diverse state spaces. This maps the agent's positional information about itself and the target trajectory point in its state to relative positional information starting from the origin. Thus, the path reasoning problem between different trajectory points is transformed into a series of similar tasks starting from the same trajectory point and reaching different target points.

[0096] During training, an experience replay pool was also used to improve the training efficiency of the agent: in each round of sampling, the state, action, reward, and state of the path reasoning agent at the current time step were tupled.<s,a,r,s’> The data is stored in the experience replay pool in the form of [data type]. During the gradient update phase, a number of data points are randomly sampled from the experience replay pool to update the gradients of the policy network, which greatly improves the efficiency of data utilization. The training process of the path search agent can be seen in [link to training process]. Figure 7 .

[0097] like Figure 4 As shown, step S40 specifically includes:

[0098] Step S41, design method to identify intelligent agents;

[0099] Similar to the pathfinding agent, at time step t in the mode identification process, the mode identification agent performs a Markov decision process based on the policy, starting from the current state s. t Take action a t Transition to the next state and receive the reward r for this moment. M (s t ,a t (wherein, the state s acquired by the agent). t This includes the local dynamic and static features of the observation sequence at that moment (including velocity, acceleration, steering angle, etc., calculated based on the temporal and spatial information of the trajectory observation sequence), road network information of the agent's location, and road network information of the surrounding grid; the action 'a' taken by the agent. t The identified mode of transportation; the reward r obtained by the agent. M (s t ,a t The value is set to the product of the road matching degree reported by the path search agent and the reward obtained based on the motion feature identification method.

[0100] Step S42, training method to identify intelligent agents;

[0101] Unlike the policy-based reinforcement learning process of the path reasoning agent, the mode recognition agent is trained using the value-based DQN algorithm. By learning the state-action value function Q(s,a), it achieves the optimal selection of the action for the traffic mode label under different states. The update rule for the agent's state-action value function Q(s,a) during each training process is as follows:

[0102] Q(s t ,a t )←Q(s t ,a t )+α[r M +γmax a’ Q(s t+1 ,a')-Q(s t ,a t )]

[0103] To effectively find the optimal Q-function, a neural network is used to model it. To analyze the differences in actions taken under the same state, the state-action value function Q is expressed as the sum of the state value function V and the advantage function A, enabling the agent to better handle action selection under similar states. The following is the expression for the state-action value function Q, where V(s) t ) represents the agent in state s t The expected return that can be obtained is A(s). t ,a t ) represents the agent in state s t Take different actions below a t The advantage function, where ω is the set of parameters of the neural network;

[0104] Q ω (s t ,a t )=V(s t )+A(s t ,a t )

[0105] To obtain deep connections between trajectory points and neighboring trajectory points, a Q-neural network is used to identify the state, action, and value of an agent. ω (s t ,a t Long Short-Term Memory (LSTM) network layers are used instead of fully connected layers. Furthermore, a target network is introduced during training. Training is performed using a dual neural network, training network Q. ω (s,a) is updated at each step during training, while the target network... Using historical parameters, the network is synchronized with the training network every C steps to avoid overestimating the network's value and negatively impacting the mode recognition performance. The training process of the mode recognition agent is shown in [link to training process]. Figure 8 .

[0106] The embodiment restores the real path of the intercity travel sparse trajectory observation data of the residents in Jiangsu Province, and part of the results are shown in Figure 9 .

[0107] The above is only a preferred embodiment of the present application, not any other form of limitation on the present application, and any modification or equivalent change made according to the technical essence of the present application still belongs to the scope claimed by the present application.

Claims

1. A multi-agent collaborative reasoning and reconstruction method for intercity multimodal travel routes (road and rail), characterized by: Includes the following steps: Step S10: Acquire and preprocess the raw trajectory observation data and traffic network data; Step S20: Rasterize and store the road network data and trajectory observation data; Step S30: Design and train a path search agent to infer a reasonable reconstructed path sequence given a trajectory observation sequence and a road network corresponding to a specific mode of transportation; Step S40: Design and train a mode recognition agent to infer a reasonable travel mode sequence based on a given trajectory observation sequence and the road matching degree fed back by the path search agent. Step S40 specifically includes: Step S41, design method to identify intelligent agents; Similar to the path-searching agent, in the mode identification process... Time step, mode identification agent performs Markov decision process according to policy, from the current state Take action Transition to the next state and receive the reward for this moment. Among them, the state acquired by the agent This includes the local dynamic and static features of the observation sequence at that moment, the road network information of the agent's location, and the road network information of the surrounding grid; the actions taken by the agent. The identified mode of transportation; the reward obtained by the intelligent agent. It is set to be the product of the road matching degree reported by the path search agent and the reward obtained based on the motion feature identification method; Step S42, training method to identify intelligent agents; Unlike the policy-based reinforcement learning process of path reasoning agents, mode recognition agents are trained using the value-based DQN algorithm, learning the state-action value function. This enables the optimal selection of traffic mode label actions under different conditions; To effectively find the optimal The function is modeled using a neural network; to analyze the differences in different actions taken under the same state, the state-action value function is... Represented as a state value function With advantage function The sum of these factors enables the agent to better handle action selection in similar states. The following formula is the state-action value function. The expression, where, For the agent in state The expected return that can be obtained is For the agent in state Take different actions The advantage function, This refers to the set of parameters for a neural network. ； To obtain deep connections between trajectory points and neighboring trajectory points, a neural network is used to identify the state, action, and value of an intelligent agent. Long Short-Term Memory (LSTM) network layers are used instead of fully connected layers. Furthermore, a target network is introduced during training. Training is performed using a dual neural network. The network is updated at each step during training, while the target network... Using historical parameters, every [time] The step is synchronized with the training network once to avoid the network overestimation affecting the method recognition performance; In step S50, the path search agent and the mode recognition agent are trained asynchronously and reason collaboratively. After reinforcement learning training is completed, they jointly realize the reasoning and reconstruction of the real path corresponding to the sparse trajectory observation sequence.

2. The intercity rail-road multimodal travel route multi-agent collaborative reasoning and reconstruction method according to claim 1, characterized in that: Step S10 specifically includes: Step S11: Acquire and preprocess trajectory observation data; The data should be obtained by acquiring trajectory observation data containing intercity travel information of travelers through specific software or mobile operators. The acquired data should at least include timestamp information and latitude and longitude information. Then, the acquired raw trajectory observation data of residents' intercity travel should be preprocessed, including data filling and deduplication operations. Step S12: Acquire and preprocess road network data; Obtain traffic network data for the study area. This data should include at least the location information of each road. Divide the overall road network data into multiple subsets based on the type of road, each corresponding to a different mode of transportation. Include complete road network data for the corresponding mode of transportation and select the subset of road data related to intercity travel for use.

3. The intercity rail-road multimodal travel route multi-agent collaborative reasoning and reconstruction method according to claim 1, characterized in that: Step S20 specifically includes: Step S21, rasterize the research area; The study area was gridded using ArcGIS software with appropriate dimensions, and the grid was then arranged according to its row and column positions as follows: Numbering is done in the form of , where The column number of the grid. The row number of the grid; Step S22: Rasterize and store the road network data; Using ArcGIS software, road networks for various modes of transportation are mapped to numbered grids. If a grid contains a road corresponding to a certain mode of transportation, the corresponding value is 1; otherwise, it is 0. After all this is done, the traffic network information for each grid is a string of numbers with a length equal to the number of modes of transportation and a value of 1 or 0. To reduce storage space, this string of 1 / 0 numbers is converted into a decimal number. At this point, the traffic network information for each grid is completely stored. With the traffic network information for each grid complete, the traffic network information for the eight surrounding grids is retrieved and stored. In other words, each grid ultimately stores the traffic network information for nine grids, including itself. The grid order is as follows: top left, top right, top right, left, itself, right, bottom left, bottom right, bottom right. Step S23: Rasterize and store trajectory observation data; The location information of the current trajectory observation data is still represented by latitude and longitude. By using ArcGIS software, all the location information in the trajectory observation data is mapped to a grid. The storage is changed from latitude and longitude to storage by the row and column of the grid. After the conversion, the trajectory observation data includes the ID, timestamp, column and row of each trajectory observation point.

4. The intercity rail-road multimodal travel route multi-agent collaborative reasoning and reconstruction method according to claim 1, characterized in that: Step S30 specifically includes: Step S31: Design a path search agent; In the path restoration process, the first At each time step, the path reasoning agent performs a Markov decision process based on the policy, starting from the current state. Take action Transition to the next state and receive the reward for this moment. Among them, the state acquired by the agent This includes road network information corresponding to the given mode of transportation, the agent's current location, and the location information of the target trajectory point; the actions taken by the agent. This is one of the action spaces, which includes movement to the upper left, directly above, upper right, directly left, directly right, lower left, directly below, and lower right; the reward obtained by the agent. The weighted sum of the follow-the-road bonus and the near-finish bonus; Step S32: Train the path search agent; The path-search agent is trained using the policy-based SAC algorithm. During training, the policy is modeled by the neural network as... , This is the set of parameters for the neural network, and the policy parameters are then determined using the gradient descent method. Learning to obtain the optimal strategy The optimization objectives are as follows: ； Since the network parameters of a single policy may not be able to adapt to the state differences between different trajectory points, goal-oriented reinforcement learning is used to enhance the agent's ability to generalize to diverse state spaces. The position information of the agent's state about itself and the target trajectory point is mapped to the relative position information starting from the origin of the coordinate system. Thus, the path reasoning problem between different trajectory points is transformed into a series of similar tasks starting from the same trajectory point and arriving at different target points. During training, an experience replay pool was also used to improve the training efficiency of the agent: in each round of sampling, the state, action, reward, and state of the path reasoning agent at the current time step were tupled. The data is stored in the experience replay pool in the form of data; during the gradient update phase, a number of data are randomly sampled from the experience replay pool to update the gradient of the policy network, which greatly improves the efficiency of data utilization.

5. The intercity rail-road multimodal travel route multi-agent collaborative reasoning and reconstruction system according to any one of claims 1-4, characterized in that: It includes modules for raw data acquisition and preprocessing, data gridding and storage, path search agent design and training, mode recognition agent design and training, and result output. The raw data acquisition and preprocessing module is used to acquire and preprocess intercity travel trajectory observation data and road network data, thereby providing a data source for subsequent rasterization processing. The data raster and storage module is used to rasterize and store road network data and observation trajectory data with implicit real paths corresponding to different modes of transportation, so as to facilitate the next step of path search and reconstruction. The path search agent design and training module is used to infer a reasonable restored path sequence based on a given trajectory observation sequence and the road network corresponding to a specific mode of transportation, so as to calculate the road matching degree as a gain information support for the training of the agent; The aforementioned identification agent design and training module is used to infer a reasonable travel mode sequence based on a given trajectory observation sequence and the road matching degree fed back by the path search agent, so as to provide the path search agent with the correct road network information and support its reasoning to reconstruct the real path corresponding to the observed trajectory. The result output module is used to, after a given trajectory observation sequence, have the trained path search agent and mode recognition agent work together to infer and reconstruct the real path and output the result.

Images