Optimization method and device for high-speed train seat distribution

Abstract

The invention provides a high-speed rail train seat allocation optimization method and device, and the method comprises the steps: obtaining the seat occupation information of a target high-speed rail train at the current moment in response to the received seat request information sent by a passenger, constructing a seat matrix representing the corresponding occupancy state of the seats of the target high-speed train distributed in each riding interval at the current moment; respectively inputting the seat matrix and the seat request information into a seat distribution model based on Markov decision so as to make a decision by using a decision sub-model corresponding to each riding interval in the seat distribution model and output a local action decision result of the riding interval; and the seat distribution model updates the seat matrix based on the local action decision result and the seat request information to obtain an updated seat matrix so as to determine a seat distribution result. Through the method, the passenger seat utilization rate and the passenger satisfaction rate of the high-speed rail train are improved, and the comprehensiveness and adaptability of seat distribution of the high-speed rail train are further improved.

Description

Technical Field

[0001] This application relates to the field of railway transportation organization technology, and in particular to an optimization method and apparatus for seat allocation on high-speed trains. Background Technology

[0002] Currently, there is still a general imbalance between supply and demand for high-speed rail seats across different time periods, routes, and regions. For example, during peak periods such as holidays and the Spring Festival travel rush, high-speed rail capacity is relatively tight, and tickets for popular high-speed rail lines are hard to come by. On weekdays or during off-peak hours, the occupancy rate of some lines is low, and a large number of seats are left unused. In addition, the different levels of development between regions also exacerbate the supply-demand mismatch. For example, some cities have high passenger demand due to their dense population and active economy, while the travel demand in other cities is relatively weak. This means that there is still considerable room for improvement in the utilization efficiency of high-speed rail resources nationwide, which reduces the comprehensiveness and adaptability of high-speed rail seat allocation.

[0003] With the continuous growth of high-speed rail passenger numbers, passengers have also raised higher requirements for the quality of railway transportation services, especially in terms of comfort and travel experience. For example, when seat allocation can fully meet passengers' personalized preferences, such as window seats, aisle seats, or adjacent seats, passenger satisfaction will be significantly improved, thereby increasing passengers' overall recognition and loyalty to railway transportation services. Thus, if existing technology cannot take into account passenger preferences when allocating seats, it will reduce passengers' high-speed rail travel experience. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide an optimization method and apparatus for high-speed train seat allocation. This method, which integrates passenger seat preferences and ticket refund response mechanisms for high-speed train operation scenarios, aims to improve the economic efficiency and service level of the high-speed rail system under limited seat resources, achieving synergistic optimization of passenger satisfaction and operational revenue. Addressing the complex situation of peak passenger arrival periods and significant heterogeneity in passenger preferences, it employs intelligent decision-making based on multi-agent deep reinforcement learning to support dynamic ticketing and refined seat allocation. This enhances the adaptability of high-speed train seat allocation to complex market demands, achieving dynamic and coordinated allocation of high-speed train seat resources, improving passenger seat utilization and satisfaction rates, and ultimately enhancing the comprehensiveness and adaptability of high-speed train seat allocation.

[0005] This application provides an optimization method for seat allocation on high-speed trains, the optimization method including: In response to receiving a passenger's seat request, obtain the seat occupancy information for the target high-speed train at the current time; Based on the seat occupancy information, a seat matrix is ​​constructed to represent the current occupancy status of the seats of the target high-speed train in each travel section. The seat matrix and the seat request information are respectively input into a pre-trained seat allocation model based on Markov decision, so that the decision sub-model corresponding to each travel segment in the seat allocation model makes a decision based on the seat request information and the seat matrix, and obtains the local action decision result corresponding to the travel segment output by each decision sub-model. The seat allocation model updates the seat matrix based on the local action decision results and the seat request information corresponding to each travel segment, thereby obtaining an updated seat matrix to determine the seat allocation result corresponding to the seat request information.

[0006] Furthermore, the step of constructing a seat matrix representing the current occupancy status of seats on the target high-speed train in each travel segment based on the seat occupancy information includes: Based on the seat occupancy information, determine the occupancy status of each seat type in each row of the target high-speed train in each preset travel section; Based on the occupancy status, a seat matrix is ​​constructed, consisting of the seat row, the travel section, and the seat type.

[0007] Furthermore, the step of inputting the seat matrix and the seat request information into a pre-trained Markov-based seat allocation model, respectively, to utilize the decision sub-model corresponding to each travel segment in the seat allocation model to make decisions based on the seat request information and the seat matrix, and to obtain the local action decision results corresponding to the travel segment output by each decision sub-model, includes: The seating area information and passenger preference information are determined from the seat request information, and a state vector is constructed based on the seat matrix and externally predicted future passenger arrival information; wherein, the state vector includes real-time seat occupancy status information, the current time step, and the future passenger arrival prediction information; The state vector is input into the decision sub-model corresponding to each travel segment in the pre-trained Markov decision-based seat allocation model; Each decision sub-model makes a decision based on the state vector and outputs the local action decision result corresponding to the travel section.

[0008] Furthermore, the step of updating the seat matrix by the seat allocation model based on the local action decision results and seat request information corresponding to each of the travel segments, to obtain an updated seat matrix, includes: Based on the predefined dictionary library of the seat allocation model, the riding interval information in the seat request information is mapped to a target column set, and the passenger preference information in the seat request information is mapped to a first seat type index; Traverse the first seat type index. When all columns in the target column set have available seats in the seat matrix, set the seat position corresponding to that row to occupied. Based on the situation where the seat position is set to occupied, output the updated seat matrix obtained by updating the seat matrix. If the first seat type index is not satisfied, then based on the passenger preference information, the target seat type in the seat matrix is ​​sequentially traversed by primary priority and secondary priority, the riding interval information is mapped to a list of preferred seat states, and the mapping of the passenger preference information is removed from the constructed candidate list to obtain the second seat type index. The second seat type index is traversed sequentially using primary priority traversal and secondary priority traversal. When all columns in the target column set correspond to available seat positions in the seat matrix and the string status of the row matches the seat status list, the seat position corresponding to the row is set to occupied, and the updated seat matrix obtained by updating the seat matrix is ​​output. When all columns in the target column set correspond to unavailable seats in the seat matrix and / or the string status of that row does not match the seat status list, traverse each row in the seat matrix according to a preset dimension order, set the traversed seats to be occupied, and output the updated seat matrix obtained by updating the seat matrix.

[0009] Furthermore, the seat allocation model is trained through the following steps: A simulation environment model is constructed, and a preset initial seat allocation model is obtained; wherein, the simulation environment model includes a high-speed rail physical model and a passenger demand model, and the initial seat allocation model includes an initial decision sub-model corresponding to each travel segment, and the initial decision sub-model includes an initial policy network model and an initial value network model; In each training cycle, the demand model generates a state vector and an initialized seat matrix corresponding to each preset simulation time step; Based on the state vector, the initialization strategy network model in the initialization decision sub-model corresponding to each of the travel segments outputs the discrete action vector corresponding to each of the travel segments; The initial seat allocation model allocates seats based on the discrete action vector corresponding to each travel segment to update the initial seat matrix, thereby obtaining a training seat matrix; Based on the discrete action vector, the training seat matrix, and the basic ticket price information and refund information generated by the high-speed rail physical model, the instant ticket purchase revenue value is determined, and based on the training seat matrix corresponding to the current ticketing process and the discrete action vector corresponding to each preset simulation time step, the empty seat penalty value is determined using the seat resource penalty attribution mechanism based on trajectory backtracking. The weighted sum between the instant ticket purchase revenue value and the empty seat penalty value is determined as the reward value for each of the initial decision sub-models at each preset simulation time step. Construct an experience tuple for each preset simulation time step, consisting of the state vector corresponding to the current preset simulation time step, the state vector corresponding to the next preset simulation time step, the discrete action vector, and the reward value. Store the experience tuple in a preset experience replay buffer to obtain an experience replay buffer dataset consisting of multiple experience tuples. Based on the experience replay buffer dataset, the initial policy network model and the initial value network model are trained for multiple training cycles using the multi-agent deep deterministic policy gradient method until the initial seat allocation model shows convergence, thus obtaining the seat allocation model.

[0010] Furthermore, the step of determining the empty seat penalty value using a trajectory backtracking-based seat resource penalty attribution mechanism, based on the training seat matrix corresponding to the current ticketing process and the discrete action vector corresponding to each preset simulation time step, includes: Traverse each seat item in the training seat matrix corresponding to the current ticketing process, and filter out seat items that are vacant throughout the journey and seat items that are partially occupied. The seat item that is vacant throughout the journey means that the seat is vacant in all travel sections. The seat item that is partially occupied means that the seat is occupied in some travel sections, but vacant in other travel sections. Based on the discrete action vector corresponding to each preset simulation time step, a first problem experience tuple and an initialization strategy network model corresponding to each attribution of a fully vacant seat item are determined in the experience playback buffer; and a second problem experience tuple and an initialization strategy network model corresponding to each attribution of a partially occupied seat item are determined. The first problem experience tuple represents the experience tuple in the experience tuple where retrospective attribution determines the existence of fully vacant seat items, and the second problem experience tuple represents the experience tuple in the experience tuple where retrospective attribution determines the existence of partially occupied seat items. The full-range fare information in the basic fare information is determined as the target fare information corresponding to the first problem experience tuple and the second problem experience tuple, respectively, and the empty seat penalty value is determined based on the target fare information.

[0011] Furthermore, based on the experience replay buffer dataset, the initial policy network model and the initial value network model are trained for multiple training cycles using the multi-agent deep deterministic policy gradient method until the initial seat allocation model shows convergence, thereby obtaining the seat allocation model, including: Multiple target experience tuples are selected from the experience replay buffer dataset. In each training cycle, the target experience tuples are input into the initialization policy network model and the initialization value network model to obtain the training action vector output by the initialization policy network model and the expected value parameter output by the initialization value network model. Based on the preset discount factor parameter, the target reward value in the target experience tuple, and the expected value parameter, calculate the first supervision signal value corresponding to the initial value network model; Based on the target state vector in the target experience tuple and the training action vector, the training prediction value corresponding to the initial value network model is calculated, and based on the loss value between the training prediction value and the first supervision signal value, the loss value is optimized by backpropagation and gradient descent to update the model parameters in the initial value network model, thereby obtaining the value network model under the training cycle. The value network model outputs the corresponding second supervision signal value based on the training action vector, and the model parameters in the initial policy network model are updated through backpropagation based on the second supervision signal value to obtain the policy network model under the training period. The value network model and its parameters are updated repeatedly over multiple training cycles until the initial seat allocation model shows convergence, thus obtaining the seat allocation model.

[0012] This application embodiment also provides an optimization device for seat allocation on high-speed trains, the optimization device comprising: The request and response module is used to respond to the seat request information sent by the passenger and obtain the seat occupancy information of the target high-speed train at the current time. The matrix construction module is used to construct a seat matrix representing the current occupancy status of the seats of the target high-speed train in each travel section based on the seat occupancy information. The model decision module is used to input the seat matrix and the seat request information into a pre-trained seat allocation model based on Markov decision, so as to use the decision sub-model corresponding to each travel segment in the seat allocation model to make a decision based on the seat request information and the seat matrix, and obtain the local action decision result corresponding to the travel segment output by each decision sub-model. The seat allocation module is used to update the seat matrix based on the local action decision results and the seat request information corresponding to each of the travel segments, so as to obtain an updated seat matrix and determine the seat allocation result corresponding to the seat request information.

[0013] Furthermore, when the matrix construction module is used to construct a seat matrix representing the occupancy status of seats on the target high-speed train in each travel segment at the current time, based on the seat occupancy information, the matrix construction module is used to: Based on the seat occupancy information, determine the occupancy status of each seat type in each row of the target high-speed train in each preset travel section; Based on the occupancy status, a seat matrix is ​​constructed, consisting of the seat row, the travel section, and the seat type.

[0014] Furthermore, when the model decision module inputs the seat matrix and the seat request information into a pre-trained Markov-based seat allocation model, and utilizes the decision sub-model corresponding to each travel segment in the seat allocation model to make decisions based on the seat request information and the seat matrix, to obtain the local action decision results corresponding to the travel segment output by each decision sub-model, the model decision module is used to: The seating area information and passenger preference information are determined from the seat request information, and a state vector is constructed based on the seat matrix and externally predicted future passenger arrival information; wherein, the state vector includes real-time seat occupancy status information, the current time step, and the future passenger arrival prediction information; The state vector is input into the decision sub-model corresponding to each travel segment in the pre-trained Markov decision-based seat allocation model; Each decision sub-model makes a decision based on the state vector and outputs the local action decision result corresponding to the travel section.

[0015] Furthermore, when the seat allocation module updates the seat matrix based on the local action decision results and seat request information corresponding to each travel segment, the seat allocation module is used to: Based on the predefined dictionary library of the seat allocation model, the riding interval information in the seat request information is mapped to a target column set, and the passenger preference information in the seat request information is mapped to a first seat type index; Traverse the first seat type index. When all columns in the target column set have available seats in the seat matrix, set the seat position corresponding to that row to occupied. Based on the situation where the seat position is set to occupied, output the updated seat matrix obtained by updating the seat matrix. If the first seat type index is not satisfied, then based on the passenger preference information, the target seat type in the seat matrix is ​​sequentially traversed by primary priority and secondary priority, the riding interval information is mapped to a list of preferred seat states, and the mapping of the passenger preference information is removed from the constructed candidate list to obtain the second seat type index. The second seat type index is traversed sequentially using primary priority traversal and secondary priority traversal. When all columns in the target column set correspond to available seat positions in the seat matrix and the string status of the row matches the seat status list, the seat position corresponding to the row is set to occupied, and the updated seat matrix obtained by updating the seat matrix is ​​output. When all columns in the target column set correspond to unavailable seats in the seat matrix and / or the string status of that row does not match the seat status list, traverse each row in the seat matrix according to a preset dimension order, set the traversed seats to be occupied, and output the updated seat matrix obtained by updating the seat matrix.

[0016] Furthermore, when the model decision module is used to train the seat allocation model, the model decision module is used to: A simulation environment model is constructed, and a preset initial seat allocation model is obtained; wherein, the simulation environment model includes a high-speed rail physical model and a passenger demand model, and the initial seat allocation model includes an initial decision sub-model corresponding to each travel segment, and the initial decision sub-model includes an initial policy network model and an initial value network model; In each training cycle, the demand model generates a state vector and an initialized seat matrix corresponding to each preset simulation time step; Based on the state vector, the initialization strategy network model in the initialization decision sub-model corresponding to each of the travel segments outputs the discrete action vector corresponding to each of the travel segments; The initial seat allocation model allocates seats based on the discrete action vector corresponding to each travel segment to update the initial seat matrix, thereby obtaining a training seat matrix; Based on the discrete action vector, the training seat matrix, and the basic ticket price information and refund information generated by the high-speed rail physical model, the instant ticket purchase revenue value is determined, and based on the training seat matrix corresponding to the current ticketing process and the discrete action vector corresponding to each preset simulation time step, the empty seat penalty value is determined using the seat resource penalty attribution mechanism based on trajectory backtracking. The weighted sum between the instant ticket purchase revenue value and the empty seat penalty value is determined as the reward value for each of the initial decision sub-models at each preset simulation time step. Construct an experience tuple for each preset simulation time step, consisting of the state vector corresponding to the current preset simulation time step, the state vector corresponding to the next preset simulation time step, the discrete action vector, and the reward value. Store the experience tuple in a preset experience replay buffer to obtain an experience replay buffer dataset consisting of multiple experience tuples. Based on the experience replay buffer dataset, the initial policy network model and the initial value network model are trained for multiple training cycles using the multi-agent deep deterministic policy gradient method until the initial seat allocation model shows convergence, thus obtaining the seat allocation model.

[0017] Furthermore, when the model decision module determines the empty seat penalty value based on the training seat matrix corresponding to the current ticketing process and the discrete action vector corresponding to each preset simulation time step using the trajectory backtracking-based seat resource penalty attribution mechanism, the model decision module is used to: Traverse each seat item in the training seat matrix corresponding to the current ticketing process, and filter out seat items that are vacant throughout the journey and seat items that are partially occupied. The seat item that is vacant throughout the journey means that the seat is vacant in all travel sections. The seat item that is partially occupied means that the seat is occupied in some travel sections, but vacant in other travel sections. Based on the discrete action vector corresponding to each preset simulation time step, a first problem experience tuple and an initialization strategy network model corresponding to each attribution of a fully vacant seat item are determined in the experience playback buffer; and a second problem experience tuple and an initialization strategy network model corresponding to each attribution of a partially occupied seat item are determined. The first problem experience tuple represents the experience tuple in the experience tuple where retrospective attribution determines the existence of fully vacant seat items, and the second problem experience tuple represents the experience tuple in the experience tuple where retrospective attribution determines the existence of partially occupied seat items. The full-range fare information in the basic fare information is determined as the target fare information corresponding to the first problem experience tuple and the second problem experience tuple, respectively, and the empty seat penalty value is determined based on the target fare information.

[0018] Furthermore, when the model decision module trains the initial policy network model and the initial value network model for multiple training cycles based on the experience replay buffer dataset using the multi-agent deep deterministic policy gradient method, until the initial seat allocation model converges and the seat allocation model is obtained, the model decision module is used to: Multiple target experience tuples are selected from the experience replay buffer dataset. In each training cycle, the target experience tuples are input into the initialization policy network model and the initialization value network model to obtain the training action vector output by the initialization policy network model and the expected value parameter output by the initialization value network model. Based on the preset discount factor parameter, the target reward value in the target experience tuple, and the expected value parameter, calculate the first supervision signal value corresponding to the initial value network model; Based on the target state vector in the target experience tuple and the training action vector, the training prediction value corresponding to the initial value network model is calculated, and based on the loss value between the training prediction value and the first supervision signal value, the loss value is optimized by backpropagation and gradient descent to update the model parameters in the initial value network model, thereby obtaining the value network model under the training cycle. The value network model outputs the corresponding second supervision signal value based on the training action vector, and the model parameters in the initial policy network model are updated through backpropagation based on the second supervision signal value to obtain the policy network model under the training period. The value network model and its parameters are updated repeatedly over multiple training cycles until the initial seat allocation model shows convergence, thus obtaining the seat allocation model.

[0019] This application also provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the optimization method for high-speed train seat allocation described above are performed.

[0020] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the above-described optimization method for high-speed train seat allocation.

[0021] The present application provides a method and apparatus for optimizing high-speed train seat allocation. The optimization method includes: in response to receiving seat request information from a passenger, obtaining seat occupancy information of a target high-speed train at the current time; based on the seat occupancy information, constructing a seat matrix representing the occupancy status of seats of the target high-speed train in each travel segment at the current time; inputting the seat matrix and the seat request information into a pre-trained Markov decision-based seat allocation model, so as to use the decision sub-model corresponding to each travel segment in the seat allocation model to make decisions based on the seat request information and the seat matrix, and obtaining the local action decision results corresponding to the travel segment output by each decision sub-model; updating the seat matrix by the seat allocation model based on the local action decision results corresponding to each travel segment and the seat request information, to obtain an updated seat matrix, thereby determining the seat allocation result corresponding to the seat request information.

[0022] Compared to existing seat allocation methods that fail to consider passenger preferences, this paper presents an optimized high-speed rail train seat allocation method that integrates passenger seat preferences and ticket refund response mechanisms within the context of high-speed rail operation. This method aims to improve the economic efficiency and service level of the high-speed rail system under limited seat resources, achieving synergistic optimization of passenger satisfaction and operational revenue. Addressing the complex scenario of peak passenger arrival times and significant heterogeneity in passenger preferences, it employs intelligent decision-making based on multi-agent deep reinforcement learning to support dynamic ticketing and refined seat allocation. This enhances the adaptability of high-speed rail train seat allocation to complex market demands, enabling dynamic and coordinated allocation of high-speed rail seat resources. Ultimately, this improves the utilization rate of high-speed rail seats and passenger satisfaction, thereby enhancing the comprehensiveness and adaptability of high-speed rail train seat allocation.

[0023] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 A flowchart illustrating an optimization method for seat allocation on a high-speed train, provided as an embodiment of this application; Figure 2 This is a schematic diagram of the mapping relationship of a seat matrix spatial structure provided in an embodiment of this application; Figure 3 A schematic diagram illustrating the backtracking identification logic and penalty path for seat resource waste provided in an embodiment of this application; Figure 4 A schematic diagram of the structure of an optimization device for seat allocation on a high-speed train provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. Based on the embodiments of this application, every other embodiment obtained by those skilled in the art without inventive effort falls within the scope of protection of this application.

[0027] Research has revealed that high-speed rail seat resources still generally suffer from an imbalance between supply and demand across time, routes, and regions. For example, during peak periods such as holidays and the Spring Festival travel rush, high-speed rail capacity is tight, and tickets for popular routes are often hard to come by. On weekdays or during off-peak hours, some routes have low occupancy rates, leaving a large number of seats unused. Furthermore, varying levels of development across regions exacerbate this supply-demand mismatch. For instance, some cities have high passenger demand due to their dense populations and active economies, while other cities have relatively weak travel demand. This results in significant room for improvement in the utilization efficiency of high-speed rail resources nationwide, reducing the comprehensiveness and adaptability of high-speed train seat allocation.

[0028] Currently, with the continuous growth of high-speed rail passengers, passengers have also put forward higher requirements for the quality of railway transportation services, especially in terms of comfort and travel experience. For example, when seat allocation can fully meet passengers' personalized preferences, such as window seats, aisle seats, or adjacent seats, passenger satisfaction will be significantly improved, thereby increasing passengers' overall recognition and loyalty to railway transportation services. Thus, if existing technology cannot take into account passenger preferences when allocating seats, it will reduce passengers' high-speed rail travel experience.

[0029] Based on this, this application provides an optimization method for high-speed train seat allocation. This method, tailored to high-speed rail operation scenarios and integrating passenger seat preferences and ticket refund response mechanisms, aims to improve the economic efficiency and service level of the high-speed rail system under limited seat resources. It achieves synergistic optimization of passenger satisfaction and operational revenue. Addressing complex scenarios with peak passenger arrival times and significant heterogeneity in passenger preferences, it employs intelligent decision-making based on multi-agent deep reinforcement learning to support dynamic ticketing and refined seat allocation. This enhances the adaptability of high-speed train seat allocation to complex market demands, enabling dynamic and coordinated allocation of high-speed rail seat resources. This improves the utilization rate of high-speed train seats and passenger satisfaction, thereby enhancing the comprehensiveness and adaptability of high-speed train seat allocation.

[0030] Specifically, at the modeling level, a three-dimensional seat matrix is ​​introduced to accurately represent the seat resource utilization status under multiple travel sections, various seat types, and high-frequency decisions. In the model decision-making method, based on a centralized training and decentralized execution architecture, a multi-agent deep deterministic policy gradient method is adopted to achieve intelligent learning of seat inventory strategies. In the action space, the operation types of "satisfying preferences" and "not satisfying preferences" are distinguished. Then, the model is trained based on a trajectory backtracking attribution reward function to take into account both immediate benefits and long-term benefits of resource allocation.

[0031] Please see Figure 1 , Figure 1 This is a flowchart illustrating an optimization method for seat allocation on a high-speed train, provided as an embodiment of this application. Figure 1 As shown in the embodiment of this application, the method for optimizing seat allocation on high-speed trains includes: S101. In response to receiving a seat request from a passenger, obtain the seat occupancy information of the target high-speed train at the current time.

[0032] In this embodiment of the application, the seat request information includes the passenger's desired travel segment on the target high-speed train, passenger preference information, and arrival time information for the ticket purchase request.

[0033] Here, the seat occupancy information of the target high-speed train at the current moment includes the occupancy status of each seat type in each row of the target high-speed train in each preset travel section.

[0034] Among them, the target high-speed train refers to the high-speed train that passengers expect to take.

[0035] S102. Based on the seat occupancy information, construct a seat matrix representing the current occupancy status of the seats of the target high-speed train in each travel section.

[0036] In this embodiment of the application, a three-dimensional seat matrix is ​​introduced as a resource representation and management structure to take into account the characteristics of the seat allocation mechanism of the high-speed rail transportation system, in order to depict the usage status of seats in high-speed trains in various travel sections and various seat types.

[0037] Here, because high-speed rail trains and air transport use different methods to control capacity based on cumulative bookings, high-speed rail operations require fixed seat allocation throughout the journey and do not allow seat changes midway. Therefore, relying solely on cumulative passenger bookings will create additional seat-changing problems, that is, although passengers are accepted, their seats cannot be used consecutively, thus violating the operational requirement of fixed seats throughout the journey.

[0038] In one possible implementation of this application, step S102 may include: S1021. Based on the seat occupancy information, determine the occupancy status of each seat type in each row of the target high-speed train in each preset travel section.

[0039] S1022. Based on the occupancy status, construct a seat matrix consisting of the seat row, the travel section, and the seat type.

[0040] In this embodiment of the application, the seat matrix is ​​represented as follows.

[0041] .

[0042] in, This indicates the occupancy status of each seat type in each row within each preset travel section; that is, the element in the seat matrix. This indicates the seating row of the target high-speed train, which is the number of rows after dividing the total number of seats of the target high-speed train according to the number of seats in each row. This indicates the preset travel section. The specific dimensions of the travel section depend on the section division of the high-speed train's operating line, such as Beijing South to Jinan West, Jinan West to Nanjing South, Nanjing South to Shanghai Hongqiao, etc. This indicates the seat type, which is consistent with the actual high-speed train seat layout (e.g., a "3+2" seat structure, i.e., A, B, C, and D, F).

[0043] Here, each element in the seat matrix represents the occupancy status of each seat in a certain travel section, and its value is a binary variable, i.e. =1 indicates that the seat is vacant, that is, the seat is available; =0 indicates that the seat has been reserved, that is, the seat has been occupied.

[0044] For example, please refer to Figure 2 , Figure 2 This is a schematic diagram illustrating the mapping relationship of a seating matrix spatial structure provided in an embodiment of this application. Figure 2 As shown, the spatial structure of the seat matrix can be represented as a three-dimensional matrix, including a preference layer (seat type), a travel interval layer, and a seat number layer.

[0045] S103. Input the seat matrix and the seat request information into a pre-trained Markov decision-based seat allocation model, so as to use the decision sub-model corresponding to each travel segment in the seat allocation model to make a decision based on the seat request information and the seat matrix, and obtain the local action decision result corresponding to the travel segment output by each decision sub-model.

[0046] In this embodiment of the application, in order to avoid the problem of policy learning difficulties caused by the exponential expansion of the action space dimension with the number of travel sections in the traditional single-model decision-making method, a distributed multi-agent structure is adopted. Each travel section is handled by a decision sub-model. Each decision sub-model only observes the state of the travel section it is responsible for and makes a locally optimal decision, thereby effectively reducing the complexity of policy learning and improving decision-making efficiency.

[0047] In one possible implementation of this application, step S103 may include: S1031. Determine the seating interval information and passenger preference information from the seat request information, and construct a state vector based on the seat matrix and externally predicted future passenger arrival information.

[0048] The state vector includes real-time seat occupancy status information, the current time step, and future passenger arrival prediction information.

[0049] In this embodiment of the application, the state space of each decision sub-model in the global state space of the seat allocation model can be represented by an integer state variable vector, and the definition of the state of each decision sub-model is as follows.

[0050] .

[0051] in, This represents the state vector of each initialized decision sub-model at each preset simulation time step; This represents the real-time seat occupancy status information determined based on the seat matrix, i.e., the passenger type corresponding to the decision sub-model. (Determined by the travel section) The number of remaining available seats can be calculated based on the seat matrix updated after each seat allocation. Indicates the current simulation time step; This represents the predicted arrival information of future passengers, that is, the passenger types from the current simulation time step until the train departs. The expected total number of passengers in the future.

[0052] Here, the state vector is used to capture information about past bookings, current time, and future events.

[0053] Here, the passenger preference information includes which seat a passenger prefers to sit in each row. For example, in a "3+2" seating layout, passengers may prefer to sit by the window (A, F), by the aisle (C, D), etc.

[0054] S1032. Input the state vector into the decision sub-model corresponding to each travel segment in the pre-trained Markov decision-based seat allocation model.

[0055] S1033. Each decision sub-model makes a decision based on the state vector and outputs the local action decision result corresponding to the travel section.

[0056] In this embodiment of the application, the local action decision result corresponding to each travel segment includes closing the seat reservation for the current travel segment, that is, rejecting the reservation request of passengers arriving within the time period; opening the seat reservation for the current travel segment, but not guaranteeing to meet the seat preferences proposed by the passengers; and opening the reservation for the current travel segment and meeting one of the seat preferences of the passengers.

[0057] In one possible implementation of this application, the step of training the seat allocation model in step S103 may include: S103A. Construct a simulation environment model and obtain the preset initial seat allocation model.

[0058] The simulation environment model includes a high-speed rail physical model and a passenger demand model. The initial seat allocation model includes an initial decision sub-model corresponding to each travel segment. The initial decision sub-model includes an initial policy network model and an initial value network model.

[0059] In this embodiment, unlike traditional single-model reinforcement learning methods, this embodiment introduces a multi-agent deep deterministic policy gradient (MADDPG) method to solve the problem. During the policy training process of each decision sub-model, the MADDPG method allows its value network model to observe global state information and joint actions with other decision sub-models, thereby effectively alleviating the non-stationarity problem in the multi-decision sub-model environment and enhancing the stability and effectiveness of policy optimization. In the actual execution stage, each decision sub-model makes decisions only based on its own local observations, achieving decentralized execution.

[0060] Specifically, each decision sub-model has an independent policy network model and a corresponding value network model. The policy network model takes the local observations of the decision sub-model itself as input and outputs a deterministic action. The value network model takes the global state and the joint actions of all decision sub-models as input and outputs the supervision signal value (action value assessment) of the corresponding decision sub-model.

[0061] S103B. In each training cycle, the demand model generates the state vector and initial seat matrix corresponding to each preset simulation time step.

[0062] The state vector includes real-time seat occupancy status information, the current simulation time step, and future passenger arrival prediction information. The state vector is used to capture past booking activities, current time, and future information.

[0063] In this embodiment of the application, the demand model uses a non-homogeneous Poisson process to simulate real passenger arrival characteristics.

[0064] S103C. The initialization strategy network model in the initialization decision sub-model corresponding to each of the travel intervals outputs the discrete action vector corresponding to each of the travel intervals based on the state vector.

[0065] In this embodiment of the application, the discrete action vector output by the initial decision sub-model belongs to the discrete action set {0, 1, 2}, where 0 indicates that seat reservations for the current travel segment are closed, i.e., the reservation requests of passengers arriving within this time period are rejected; 1 indicates that seat reservations for the current travel segment are open, but the seat preferences proposed by the passengers are not guaranteed to be met; and 2 indicates that reservations for the current travel segment are open and the seat preferences of the passengers are met.

[0066] Thus, the action space design adopted in this application embodiment has good scalability. The action space dimension does not increase with the number of agents (i.e., the number of train sections), thereby effectively avoiding the problem of increased policy learning difficulty caused by the expansion of the action dimension. This structural design ensures that training can still be carried out stably and efficiently when facing larger-scale train operation lines and more section divisions, which is conducive to the model adapting to high-speed rail operation scenarios of different scales in actual deployment.

[0067] S103D: The initial seat allocation model allocates seats based on the discrete action vector corresponding to each travel segment to update the initial seat matrix and obtain the training seat matrix.

[0068] In this embodiment, the initial seat allocation model makes a global decision based on the local discrete action vectors of each initial decision sub-model to update the initial seat matrix and obtain the training seat matrix.

[0069] S103E: Based on the discrete action vector, the training seat matrix, and the basic ticket price information and refund information generated by the high-speed rail physical model, determine the instant ticket purchase revenue value, and based on the training seat matrix corresponding to the current ticketing process and the discrete action vector corresponding to each preset simulation time step, use the seat resource penalty attribution mechanism based on trajectory backtracking to determine the empty seat penalty value.

[0070] In this embodiment of the application, the instant ticket purchase revenue value is calculated using the following formula.

[0071] .

[0072] in, This represents the instantaneous ticket purchase revenue value at the current simulation time step; This indicates the number of passengers who were successfully assigned seats at the current simulation time step; Indicates passenger type The base fare for the corresponding travel segment; This represents the revenue adjustment coefficient determined based on whether passenger preferences are met, which depends on the actions and outcomes performed by the decision sub-model; This indicates the revenue loss resulting from subsequent ticket refunds by passengers.

[0073] For example, if discrete action vectors are used and seat preferences are satisfied, then If discrete action vectors are used, then If discrete action vectors are used but seat preferences are not satisfied, then If the designated travel area is not open, then .

[0074] For example, if a passenger's ticket order is cancelled in the future, then If the passenger's ticket order is not cancelled in the future, then .

[0075] Furthermore, to further improve the utilization efficiency of seat resources and rationally guide the seat allocation model to weigh immediate and long-term benefits during the ticketing process, avoiding resource waste caused by short-sighted decisions, a seat resource penalty attribution mechanism based on trajectory backtracking is adopted. After the simulation process, the state of the seat matrix is ​​systematically backtracked and analyzed to identify potential resource waste behaviors and trace back to the key decision points that led to the behavior. Penalties are imposed on the relevant decision sub-models, thereby improving the global optimal performance of policy learning. The core idea of ​​this mechanism is: for seat resources that are not effectively utilized after the entire booking period, the reasons for resource vacancy are identified through trajectory backtracking, and negative rewards linked to opportunity costs are imposed on the corresponding responsible parties (decision sub-models) according to the specific type of seat vacancy.

[0076] In one possible implementation of this application, in specific implementation, the step S103E of determining the empty seat penalty value based on the training seat matrix and the discrete action vector using a trajectory backtracking-based seat resource penalty attribution mechanism may include: S103E1, Traverse each seat item in the training seat matrix corresponding to the current ticketing process, and filter out seat items that are vacant throughout the process and seat items that are partially occupied.

[0077] The term "fully vacant seat" indicates that the seat is vacant throughout the entire travel segment; the term "partially occupied seat" indicates that the seat is occupied in some travel segments, but vacant in others.

[0078] S103E2. Based on the discrete action vector corresponding to each preset simulation time step, determine in the experience playback buffer the first problem experience tuple corresponding to each of the attributions of the fully vacant seats and the initialization strategy network model corresponding to the first problem experience tuple, as well as the second problem experience tuple corresponding to each of the attributions of the locally occupied seats and the initialization strategy network model corresponding to the second problem experience tuple.

[0079] Wherein, the first problem experience set represents the problem experience set in the experience set that retrospective attribution determines that there are completely vacant seats, and the second problem experience set represents the problem experience set in the experience set that retrospective attribution determines that there are partially occupied seats.

[0080] In this embodiment of the application, for the first problem experience tuple corresponding to the attribution of the vacant seat type, all actions that "reject" passenger requests within the travel section (denoted as action 0) are backtracked, and a fixed intensity of penalty is imposed on these decision nodes.

[0081] Here, the partial occupancy of seats reflects the encroachment of short-term, low-return orders on seat resources, thus hindering the fulfillment of higher-return, long-distance orders and consequently reducing resource utilization efficiency. To trace the responsible party (decision sub-model), we first identify the fact that the seat is occupied only within a specific range, then trace back the order record that generated the occupancy behavior, and attribute it to the decision sub-model that accepted the order, imposing a corresponding opportunity cost penalty.

[0082] S103E3, The full-range fare information in the basic fare information is determined as the target fare information corresponding to the first problem experience tuple and the second problem experience tuple, respectively, and the empty seat penalty value is determined based on the target fare information.

[0083] S103F, The weighted sum between the instant ticket purchase revenue value and the empty seat penalty value is determined as the reward value for each of the initial decision sub-models at each preset simulation time step.

[0084] In this application, the weighted sum between the instant ticket purchase revenue value and the empty seat penalty value is calculated using the following formula, which is the reward value corresponding to each initial decision sub-model at each preset simulation time step.

[0085] .

[0086] Wherein, it represents the reward value of each initial decision sub-model at each preset simulation time step; This represents the immediate revenue from ticket purchases. This represents the penalty value for an empty seat; and These represent the corresponding weights.

[0087] S103G: Construct an experience tuple for each preset simulation time step, consisting of the state vector corresponding to the current preset simulation time step, the state vector corresponding to the next preset simulation time step, the discrete action vector, and the reward value, and store the experience tuple in a preset experience replay buffer to obtain an experience replay buffer dataset consisting of multiple experience tuples.

[0088] Here, an experience replay buffer is constructed to store experience tuples, and historical experience tuples are reused for updates, reducing the training instability caused by sample correlation. Each experience data is stored in the form of a quadruple.

[0089] In this embodiment, the experience tuple includes the state vector corresponding to the current preset simulation time step, the state vector corresponding to the next preset simulation time step (determined based on the training seat matrix), the reward value, and the discrete action vector.

[0090] S103H: Based on the experience replay buffer dataset, the initial policy network model and the initial value network model are trained for multiple training cycles using the multi-agent deep deterministic policy gradient method until the initial seat allocation model shows convergence, thus obtaining the seat allocation model.

[0091] In this embodiment of the application, in order to further enhance the stability of the training process and alleviate update noise, a target network mechanism is introduced. That is, a target network is maintained for each initial policy network model and initial value network model of the initial decision sub-model to generate stable target values. During the training process, the parameters of the target network are not immediately replaced with the parameters of the current network, but are updated in a soft manner, that is, the parameters of the main network are slowly approximated with a small step size.

[0092] Here, since the actions of each agent in this embodiment are discrete variables (such as "whether to open the ticket window" and "whether to satisfy seat preferences"), this embodiment introduces the Gumbel-Softmax reparameterization method during the training phase to achieve differentiable approximate sampling of discrete actions, thereby maintaining the gradient transferability of the policy network.

[0093] The Gumbel-Softmax reparameterization method introduces Gumbel distribution sampling noise into the output of the policy network model and combines it with the Softmax function to generate a continuously differentiable approximate probability vector. Then, it generates the final discrete action vector through a "hardening" operation (such as one-hot maximization). This achieves a sampling mechanism that balances differentiability and policy execution. The temperature parameter in Gumbel-Softmax can adjust the approximation of the sampling distribution: the lower the temperature, the closer the output is to the standard one-hot form, thereby improving policy determinism; the higher the temperature, the more randomness and exploration ability of the policy are promoted. In this way, by setting a higher temperature in the early stage of training to enhance policy exploration and gradually lowering the temperature in the later stage to stabilize the output behavior, smooth learning and efficient optimization in the discrete action space are achieved.

[0094] In this step, during each training cycle, all initial decision sub-models first interact with the simulation environment model to collect a series of empirical tuples, which are then stored in the experience replay buffer. Subsequently, a certain number of sample batches are randomly selected from the experience replay buffer to update the neural network parameters. By minimizing the temporal difference (TD) error, the parameters of the initial value network model are optimized. Next, based on the updated value network model, the corresponding policy network model is updated using the policy gradient method. After completing the parameter update for the current batch, all target networks also undergo a simultaneous soft update to continuously track the evolution of the main network. This process is iterated repeatedly until the overall policy converges.

[0095] In one possible implementation of this application, step S103H may include: S103H1. Select multiple target experience tuples from the experience replay buffer dataset. In each training cycle, input the target experience tuples into the initialization policy network model and the initialization value network model to obtain the training action vector output by the initialization policy network model and the expected value parameter output by the initialization value network model.

[0096] S103H2, based on the preset discount factor parameter, the target reward value in the target experience tuple and the expected value parameter, calculate the first supervision signal value corresponding to the initial value network model.

[0097] S103H3. Based on the target state vector in the target experience tuple and the training action vector, calculate the training prediction value corresponding to the initial value network model, and optimize the loss value based on the loss value between the training prediction value and the first supervision signal value through backpropagation and gradient descent to update the model parameters in the initial value network model, thereby obtaining the value network model under the training cycle.

[0098] S103H4. Using the value network model, output the corresponding second supervision signal value based on the training action vector, and based on the second supervision signal value, update the model parameters in the initial policy network model through backpropagation to obtain the policy network model under the training period.

[0099] S103H5, Repeat the update of the value network model and its parameters for multiple training cycles until the initial seat allocation model shows convergence, thus obtaining the seat allocation model.

[0100] For example, please refer to Figure 3 , Figure 3 This diagram illustrates a backtracking identification logic and penalty path for addressing wasted seating resources, as provided in an embodiment of this application. Figure 3 As shown, Figure 3 It reveals an offline attribution and feedback mechanism that breaks away from the limitations of single-step decision-making. It performs a global audit after a complete business cycle, achieves accurate accountability through order logs, and corrects the learning direction of the agent by modifying historical experience data.

[0101] Specifically, such as Figure 3 As shown, the first stage is simulation and environment recording. The system runs a complete ticket sales cycle (from the start of ticket sales to the train's departure). During this process, the decision sub-model interacts with the environment based on the current strategy to generate a series of decision actions. The system executes these actions, updates the state, records the real-time reward for each step, and outputs the final seat matrix, decision sequence, and order information log.

[0102] The order information log records in detail the time, passenger type, preferences, specific seat number assigned, and which agent handled each successful transaction. This is crucial evidence for subsequent accountability.

[0103] Furthermore, such as Figure 3 As shown, the second stage is problem identification and accountability. After the simulation, the system analyzes the final seat matrix to identify seats where resources are wasted. For seats that are empty throughout the journey, the system backtracks the order information to find all rejected long-distance passenger requests that covered the seat during the entire simulation period. The decision points where these requests were rejected (action a=0) are the responsible parties for the vacancy. For seats that are partially occupied (e.g., only segments AB were sold, while segments BC and CD are empty), the system uses the order information log to directly locate which order (which decision sub-model and when) allocated the seat to that short-distance passenger. This allocation decision is the responsible party for the subsequent vacancy.

[0104] Furthermore, such as Figure 3 As shown in the diagram, the third stage is the execution of punishment. Once the responsibility decision is determined, the system locates the corresponding specific record in the temporary experience pool and modifies the reward value R in that record. k The original instant reward R k Updated to R k ', that is, R k ' = R k -β×(longest interval ticket price), where β is the penalty weight.

[0105] Furthermore, such as Figure 3As shown, the fourth stage is that the updated experience (records with penalties) after closed-loop learning is stored in the total experience replay buffer. In subsequent reinforcement learning training, the decision sub-model will sample these experiences. In this way, when the value network model evaluates the value of these actions, it will give a lower rating because the reward includes penalties. The decision network model will then learn to avoid making decisions that would lead to the same waste of resources in the future through policy gradient updates.

[0106] S104. The seat allocation model updates the seat matrix based on the local action decision results and the seat request information corresponding to each of the travel segments to obtain an updated seat matrix, thereby determining the seat allocation result corresponding to the seat request information.

[0107] In this step, in specific implementation, firstly, the seat allocation model determines the ticketing strategy and seat allocation method corresponding to each travel segment corresponding to the seat request information based on the local action decision results output by each decision sub-model for each travel segment; then, the seat allocation method is used to allocate seats in response to the seat request information, and the updated seat matrix is ​​determined after seat allocation based on the seat matrix; finally, the seat allocation result including the seat allocation method and the updated seat matrix is ​​determined.

[0108] In this embodiment of the application, the method for seat allocation in the seat allocation model is a preference-response-based seat allocation mechanism, which includes two stages: priority matching and resource reservation.

[0109] In the priority matching phase, if the decision is to satisfy the passenger's preferences, the system prioritizes searching for consecutive empty seats that meet the criteria under the preference dimension and completes the seat allocation. In the resource reservation phase, if the preferences cannot be satisfied or the decision is not to satisfy the passenger's preferences, the system adopts the "replenishment priority" strategy, that is, it prioritizes using seat resources that have been partially occupied in other sections to avoid creating new complete empty seats, so as to reserve more resources for long-distance high-value passengers.

[0110] For example, taking a short-distance passenger traveling from Beijing South Station to Jinan West Station as an example, firstly, find seats that are not yet assigned only in the section from Beijing South Station to Jinan West Station, but have been used by other passengers in the subsequent sections (Jinan West Station to Nanjing South Station and Nanjing South Station to Shanghai Hongqiao Station); if the above conditions cannot be met, then find seats that are not assigned in the target section, and exactly one of the subsequent two sections has been assigned; if this still cannot be met, then select completely empty seats that are not assigned in any of the three sections.

[0111] In one possible implementation of this application, in specific implementation, step S104, where the seat allocation model updates the seat matrix based on the local action decision results and seat request information corresponding to each of the travel segments, may include: S1041. Based on the predefined dictionary library of the seat allocation model, map the seating interval information in the seat request information to a target column set, and map the passenger preference information in the seat request information to a first seat type index.

[0112] S1042. Traverse the first seat type index. When all columns in the target column set have available seats in the seat matrix, set the seat position corresponding to that row to occupied. Based on the situation where the seat position is set to occupied, output the updated seat matrix obtained by updating the seat matrix.

[0113] S1043. If the first seat type index is not satisfied, then based on the passenger preference information, the target seat type in the seat matrix is ​​traversed sequentially using primary priority traversal and secondary priority traversal.

[0114] S1044. Map the riding interval information to a preferred seat status list, and remove the mapping of the passenger preference information from the constructed candidate list to obtain a second seat type index.

[0115] S1045. Perform primary priority traversal and secondary priority traversal on the second seat type index in sequence. When all columns in the target column set have available seats in the seat matrix and the string status of the row matches the seat status list, set the seat position corresponding to the row to occupied, and output the updated seat matrix obtained by updating the seat matrix.

[0116] S1046. When all columns in the target column set correspond to unavailable seats in the seat matrix and / or the string status of the row does not match the seat status list, traverse each row in the seat matrix according to a preset dimension order, set the traversed seats to be occupied, and output the updated seat matrix obtained by updating the seat matrix.

[0117] For example, when passenger preferences cannot be met, the travel interval is "abcd". The travel interval "ab" is represented as 11 and the travel interval "ac" is represented as 12. For example, first, the seat type B is traversed first, and then traversed in order of primary priority and secondary priority.

[0118] In the primary-priority traversal mapping, in addition to the strings "1, 2, 3", each passenger type corresponds to a string, and the string corresponds to the travel interval status, "1" means available): - '11': "100"; - '22': "010"; - '33': "001"; - '12': "110"; - '23': "011".

[0119] In the second-priority traversal mapping, the second-priority traversal mapping is only applicable to passenger types '11', '22', '33': - '11': ["110"]; - '22': ["110", "011"]; - '33': ["011"].

[0120] Furthermore, if none of the seat types B can be assigned, then the candidate seat types other than seat B are traversed, first traversing the candidate seats in the primary priority mode; then for passenger types '11', '22', and '33', if the primary priority mode is not satisfied, then the secondary priority mode is traversed.

[0121] Furthermore, if none of the above conditions are met, then all rows and all dimensions are traversed in the order of "seats corresponding to passenger preferences, other seats"; if no seats can be assigned after traversing all rows, then the revenue is returned as 0 and the order information is None.

[0122] Thus, the high-speed train seat allocation optimization method provided in this application realizes the optimized allocation of seat inventory in high-speed rail operations by integrating passenger preferences and ticket refund behavior, and has the following beneficial effects: First, a three-dimensional seat matrix based on "seat row number - travel section - seat type" is constructed, which can accurately represent the dynamic occupancy status of train seat resources under multiple sections and multiple passenger preferences; then, in the seat allocation process, the operation types of "satisfying preferences" and "not satisfying preferences" are distinguished, which significantly improves the model's adaptability to personalized services; in addition, a multi-agent deep reinforcement learning method is adopted to realize intelligent decision-making for high-speed rail seat inventory control, which does not rely on accurate market demand or passenger behavior models, but directly learns the optimal strategy through continuous interaction with the environment, which not only effectively copes with the challenges brought by high-dimensional states and action spaces, but also provides a new solution path for the optimization of complex dynamic systems; finally, an attribution reward function is designed, which combines real-time ticket price revenue and trajectory backtracking penalty, strengthens the consideration of long-term resource allocation efficiency, guides the decision sub-model to make a reasonable trade-off between short-term gains and long-term resource utilization, and achieves the overall optimal system revenue.

[0123] The high-speed train seat allocation optimization method provided in this application aims to improve the economic efficiency and service level of the high-speed rail system under limited seat resources. This method integrates passenger seat preferences and ticket refund behavior response mechanisms, and is designed for high-speed rail operation scenarios. It achieves synergistic optimization of passenger satisfaction and operating revenue. For complex scenarios with peak passenger arrival periods and significant heterogeneity in passenger preferences, it employs intelligent decision-making based on multi-agent deep reinforcement learning to support dynamic ticketing and refined seat allocation. This enhances the adaptability of high-speed train seat allocation to complex market demands, achieves dynamic and coordinated allocation of high-speed rail seat resources, improves the utilization rate of high-speed train seats and passenger satisfaction, and thus improves the comprehensiveness and adaptability of high-speed train seat allocation.

[0124] Please see Figure 4 , Figure 4 This is a schematic diagram of a high-speed train seat allocation optimization device provided in an embodiment of this application. Figure 4 As shown, the optimization device 400 includes: The request response module 410 is used to respond to the seat request information sent by the passenger and obtain the seat occupancy information of the target high-speed train at the current time. The matrix construction module 420 is used to construct a seat matrix representing the current occupancy status of the seats of the target high-speed train in each travel section based on the seat occupancy information. The model decision module 430 is used to input the seat matrix and the seat request information into a pre-trained seat allocation model based on Markov decision, so as to use the decision sub-model corresponding to each travel segment in the seat allocation model to make a decision based on the seat request information and the seat matrix, and obtain the local action decision result corresponding to the travel segment output by each decision sub-model. The seat allocation module 440 is used to update the seat matrix based on the local action decision results corresponding to each of the travel segments and the seat request information, so as to obtain an updated seat matrix and determine the seat allocation result corresponding to the seat request information.

[0125] Furthermore, when the matrix construction module 420 constructs a seat matrix representing the occupancy status of seats on the target high-speed train in each travel segment at the current time, based on the seat occupancy information, the matrix construction module 420 is used to: Based on the seat occupancy information, determine the occupancy status of each seat type in each row of the target high-speed train in each preset travel section; Based on the occupancy status, a seat matrix is ​​constructed, consisting of the seat row, the travel section, and the seat type.

[0126] Furthermore, when the model decision module 430 inputs the seat matrix and the seat request information into a pre-trained Markov-based seat allocation model, and utilizes the decision sub-model corresponding to each travel segment in the seat allocation model to make a decision based on the seat request information and the seat matrix, to obtain the local action decision result corresponding to the travel segment output by each decision sub-model, the model decision module 430 is used to: The seating area information and passenger preference information are determined from the seat request information, and a state vector is constructed based on the seat matrix and externally predicted future passenger arrival information; wherein, the state vector includes real-time seat occupancy status information, the current time step, and the future passenger arrival prediction information; The state vector is input into the decision sub-model corresponding to each travel segment in the pre-trained Markov decision-based seat allocation model; Each decision sub-model makes a decision based on the state vector and outputs the local action decision result corresponding to the travel section.

[0127] Furthermore, when the seat allocation module 440 updates the seat matrix based on the local action decision results and seat request information corresponding to each of the travel segments, to obtain an updated seat matrix, the seat allocation module 440 is used to: Based on the predefined dictionary library of the seat allocation model, the riding interval information in the seat request information is mapped to a target column set, and the passenger preference information in the seat request information is mapped to a first seat type index; Traverse the first seat type index. When all columns in the target column set have available seats in the seat matrix, set the seat position corresponding to that row to occupied. Based on the situation where the seat position is set to occupied, output the updated seat matrix obtained by updating the seat matrix. If the first seat type index is not satisfied, then based on the passenger preference information, the target seat type in the seat matrix is ​​sequentially traversed by primary priority and secondary priority, the riding interval information is mapped to a list of preferred seat states, and the mapping of the passenger preference information is removed from the constructed candidate list to obtain the second seat type index. The second seat type index is traversed sequentially using primary priority traversal and secondary priority traversal. When all columns in the target column set correspond to available seat positions in the seat matrix and the string status of the row matches the seat status list, the seat position corresponding to the row is set to occupied, and the updated seat matrix obtained by updating the seat matrix is ​​output. When all columns in the target column set correspond to unavailable seats in the seat matrix and / or the string status of that row does not match the seat status list, traverse each row in the seat matrix according to a preset dimension order, set the traversed seats to be occupied, and output the updated seat matrix obtained by updating the seat matrix.

[0128] Furthermore, when training the seat allocation model, the model decision module 430 is used to: A simulation environment model is constructed, and a preset initial seat allocation model is obtained; wherein, the simulation environment model includes a high-speed rail physical model and a passenger demand model, and the initial seat allocation model includes an initial decision sub-model corresponding to each travel segment, and the initial decision sub-model includes an initial policy network model and an initial value network model; In each training cycle, the demand model generates a state vector and an initialized seat matrix corresponding to each preset simulation time step; Based on the state vector, the initialization strategy network model in the initialization decision sub-model corresponding to each of the travel segments outputs the discrete action vector corresponding to each of the travel segments; The initial seat allocation model allocates seats based on the discrete action vector corresponding to each travel segment to update the initial seat matrix, thereby obtaining a training seat matrix; Based on the discrete action vector, the training seat matrix, and the basic ticket price information and refund information generated by the high-speed rail physical model, the instant ticket purchase revenue value is determined, and based on the training seat matrix corresponding to the current ticketing process and the discrete action vector corresponding to each preset simulation time step, the empty seat penalty value is determined using the seat resource penalty attribution mechanism based on trajectory backtracking. The weighted sum between the instant ticket purchase revenue value and the empty seat penalty value is determined as the reward value for each of the initial decision sub-models at each preset simulation time step. Construct an experience tuple for each preset simulation time step, consisting of the state vector corresponding to the current preset simulation time step, the state vector corresponding to the next preset simulation time step, the discrete action vector, and the reward value. Store the experience tuple in a preset experience replay buffer to obtain an experience replay buffer dataset consisting of multiple experience tuples. Based on the experience replay buffer dataset, the initial policy network model and the initial value network model are trained for multiple training cycles using the multi-agent deep deterministic policy gradient method until the initial seat allocation model shows convergence, thus obtaining the seat allocation model.

[0129] Furthermore, when the model decision module 430 determines the empty seat penalty value based on the training seat matrix corresponding to the current ticketing process and the discrete action vector corresponding to each preset simulation time step using the seat resource penalty attribution mechanism based on trajectory backtracking, the model decision module 430 is used to: Traverse each seat item in the training seat matrix corresponding to the current ticketing process, and filter out seat items that are vacant throughout the journey and seat items that are partially occupied. The seat item that is vacant throughout the journey means that the seat is vacant in all travel sections. The seat item that is partially occupied means that the seat is occupied in some travel sections, but vacant in other travel sections. Based on the discrete action vector corresponding to each preset simulation time step, a first problem experience tuple and an initialization strategy network model corresponding to each attribution of a fully vacant seat item are determined in the experience playback buffer; and a second problem experience tuple and an initialization strategy network model corresponding to each attribution of a partially occupied seat item are determined. The first problem experience tuple represents the experience tuple in the experience tuple where retrospective attribution determines the existence of fully vacant seat items, and the second problem experience tuple represents the experience tuple in the experience tuple where retrospective attribution determines the existence of partially occupied seat items. The full-range fare information in the basic fare information is determined as the target fare information corresponding to the first problem experience tuple and the second problem experience tuple, respectively, and the empty seat penalty value is determined based on the target fare information.

[0130] Furthermore, when the model decision module 430 trains the initial policy network model and the initial value network model for multiple training cycles based on the experience replay buffer dataset using the multi-agent deep deterministic policy gradient method, until the initial seat allocation model converges and the seat allocation model is obtained, the model decision module 430 is used to: Multiple target experience tuples are selected from the experience replay buffer dataset. In each training cycle, the target experience tuples are input into the initialization policy network model and the initialization value network model to obtain the training action vector output by the initialization policy network model and the expected value parameter output by the initialization value network model. Based on the preset discount factor parameter, the target reward value in the target experience tuple, and the expected value parameter, calculate the first supervision signal value corresponding to the initial value network model; Based on the target state vector in the target experience tuple and the training action vector, the training prediction value corresponding to the initial value network model is calculated, and based on the loss value between the training prediction value and the first supervision signal value, the loss value is optimized by backpropagation and gradient descent to update the model parameters in the initial value network model, thereby obtaining the value network model under the training cycle. The value network model outputs the corresponding second supervision signal value based on the training action vector, and the model parameters in the initial policy network model are updated through backpropagation based on the second supervision signal value to obtain the policy network model under the training period. The value network model and its parameters are updated repeatedly over multiple training cycles until the initial seat allocation model shows convergence, thus obtaining the seat allocation model.

[0131] The high-speed train seat allocation optimization device provided in this application embodiment, through an optimization method for high-speed train seat allocation that is oriented towards high-speed rail operation scenarios and integrates passenger seat preferences and ticket refund behavior response mechanisms, aims to improve the economic efficiency and service level of the high-speed rail system under limited seat resources, and achieve synergistic optimization of passenger satisfaction and operating revenue. For complex scenarios where passenger arrivals are at peak times and passenger preferences are significantly heterogeneous, it employs intelligent decision-making based on multi-agent deep reinforcement learning to support dynamic ticketing and refined seat allocation, improving the adaptability of high-speed train seat allocation to complex market demands, realizing dynamic and coordinated allocation of high-speed rail seat resources, increasing high-speed train seat utilization and passenger satisfaction, and thus improving the comprehensiveness and adaptability of high-speed train seat allocation.

[0132] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 5 As shown, the electronic device 500 includes a processor 510, a memory 520, and a bus 530.

[0133] The memory 520 stores machine-readable instructions executable by the processor 510. When the electronic device 500 is running, the processor 510 and the memory 520 communicate via the bus 530. When the machine-readable instructions are executed by the processor 510, they can perform the operations described above. Figure 1 The steps of the optimization method for high-speed train seat allocation in the method embodiment shown are described in detail in the method embodiment, and will not be repeated here.

[0134] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the above-described actions. Figure 1 The steps of the optimization method for high-speed train seat allocation in the method embodiment shown are described in detail in the method embodiment, and will not be repeated here.

[0135] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0136] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0137] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0138] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0139] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0140] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The scope of protection of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An optimization method for seat allocation on high-speed trains, characterized in that, The optimization method includes: In response to receiving a passenger's seat request, obtain the seat occupancy information for the target high-speed train at the current time; Based on the seat occupancy information, a seat matrix is ​​constructed to represent the current occupancy status of the seats of the target high-speed train in each travel section. The seat matrix and the seat request information are respectively input into a pre-trained seat allocation model based on Markov decision, so that the decision sub-model corresponding to each travel segment in the seat allocation model makes a decision based on the seat request information and the seat matrix, and obtains the local action decision result corresponding to the travel segment output by each decision sub-model. The seat allocation model updates the seat matrix based on the local action decision results and the seat request information corresponding to each travel segment, thereby obtaining an updated seat matrix to determine the seat allocation result corresponding to the seat request information.

2. The method according to claim 1, characterized in that, The step of constructing a seat matrix based on the seat occupancy information, representing the current occupancy status of seats on the target high-speed train in each travel segment, includes: Based on the seat occupancy information, determine the occupancy status of each seat type in each row of the target high-speed train in each preset travel section; Based on the occupancy status, a seat matrix is ​​constructed, consisting of the seat row, the travel section, and the seat type.

3. The method according to claim 1, characterized in that, The step involves inputting the seat matrix and the seat request information into a pre-trained Markov-based seat allocation model, respectively. This allows the decision sub-model corresponding to each travel segment in the seat allocation model to make decisions based on the seat request information and the seat matrix, obtaining local action decision results for each travel segment output by each decision sub-model. This includes: The seating area information and passenger preference information are determined from the seat request information, and a state vector is constructed based on the seat matrix and externally predicted future passenger arrival information; wherein, the state vector includes real-time seat occupancy status information, the current time step, and the future passenger arrival prediction information; The state vector is input into the decision sub-model corresponding to each travel segment in the pre-trained Markov decision-based seat allocation model; Each decision sub-model makes a decision based on the state vector and outputs the local action decision result corresponding to the travel section.

4. The method according to claim 1, characterized in that, The step of updating the seat matrix by the seat allocation model based on the local action decision results and seat request information corresponding to each travel segment, to obtain an updated seat matrix, includes: Based on the predefined dictionary library of the seat allocation model, the riding interval information in the seat request information is mapped to a target column set, and the passenger preference information in the seat request information is mapped to a first seat type index; Traverse the first seat type index. When all columns in the target column set have available seats in the seat matrix, set the seat position corresponding to that row to occupied. Based on the situation where the seat position is set to occupied, output the updated seat matrix obtained by updating the seat matrix. If the first seat type index is not satisfied, then based on the passenger preference information, the target seat type in the seat matrix is ​​sequentially traversed by primary priority and secondary priority, the riding interval information is mapped to a list of preferred seat states, and the mapping of the passenger preference information is removed from the constructed candidate list to obtain the second seat type index. The second seat type index is traversed sequentially using primary priority traversal and secondary priority traversal. When all columns in the target column set correspond to available seat positions in the seat matrix and the string status of the row matches the seat status list, the seat position corresponding to the row is set to occupied, and the updated seat matrix obtained by updating the seat matrix is ​​output. When all columns in the target column set correspond to unavailable seats in the seat matrix and / or the string status of that row does not match the seat status list, traverse each row in the seat matrix according to a preset dimension order, set the traversed seats to be occupied, and output the updated seat matrix obtained by updating the seat matrix.

5. The method according to claim 1, characterized in that, The seat allocation model is trained using the following steps: A simulation environment model is constructed, and a preset initial seat allocation model is obtained; wherein, the simulation environment model includes a high-speed rail physical model and a passenger demand model, and the initial seat allocation model includes an initial decision sub-model corresponding to each travel segment, and the initial decision sub-model includes an initial policy network model and an initial value network model; In each training cycle, the demand model generates a state vector and an initialized seat matrix corresponding to each preset simulation time step; Based on the state vector, the initialization strategy network model in the initialization decision sub-model corresponding to each of the travel segments outputs the discrete action vector corresponding to each of the travel segments; The initial seat allocation model allocates seats based on the discrete action vector corresponding to each travel segment to update the initial seat matrix, thereby obtaining a training seat matrix; Based on the discrete action vector, the training seat matrix, and the basic ticket price information and refund information generated by the high-speed rail physical model, the instant ticket purchase revenue value is determined, and based on the training seat matrix corresponding to the current ticketing process and the discrete action vector corresponding to each preset simulation time step, the empty seat penalty value is determined using the seat resource penalty attribution mechanism based on trajectory backtracking. The weighted sum between the instant ticket purchase revenue value and the empty seat penalty value is determined as the reward value for each of the initial decision sub-models at each preset simulation time step. Construct an experience tuple for each preset simulation time step, consisting of the state vector corresponding to the current preset simulation time step, the state vector corresponding to the next preset simulation time step, the discrete action vector, and the reward value. Store the experience tuple in a preset experience replay buffer to obtain an experience replay buffer dataset consisting of multiple experience tuples. Based on the experience replay buffer dataset, the initial policy network model and the initial value network model are trained for multiple training cycles using the multi-agent deep deterministic policy gradient method until the initial seat allocation model shows convergence, thus obtaining the seat allocation model.

6. The method according to claim 5, characterized in that, The process of determining the empty seat penalty value using a trajectory backtracking-based seat resource penalty attribution mechanism, based on the training seat matrix corresponding to the current ticketing process and the discrete action vector corresponding to each preset simulation time step, includes: Traverse each seat item in the training seat matrix corresponding to the current ticketing process, and filter out seat items that are vacant throughout the journey and seat items that are partially occupied. The seat item that is vacant throughout the journey means that the seat is vacant in all travel sections. The seat item that is partially occupied means that the seat is occupied in some travel sections, but vacant in other travel sections. Based on the discrete action vector corresponding to each preset simulation time step, a first problem experience tuple and an initialization strategy network model corresponding to each attribution of a fully vacant seat item are determined in the experience playback buffer; and a second problem experience tuple and an initialization strategy network model corresponding to each attribution of a partially occupied seat item are determined. The first problem experience tuple represents the experience tuple in the experience tuple where retrospective attribution determines the existence of fully vacant seat items, and the second problem experience tuple represents the experience tuple in the experience tuple where retrospective attribution determines the existence of partially occupied seat items. The full-range fare information in the basic fare information is determined as the target fare information corresponding to the first problem experience tuple and the second problem experience tuple, respectively, and the empty seat penalty value is determined based on the target fare information.

7. The method according to claim 5, characterized in that, Based on the experience replay buffer dataset, the initial policy network model and the initial value network model are trained for multiple training cycles using the multi-agent deep deterministic policy gradient method until the initial seat allocation model shows convergence, thus obtaining the seat allocation model, including: Multiple target experience tuples are selected from the experience replay buffer dataset. In each training cycle, the target experience tuples are input into the initialization policy network model and the initialization value network model to obtain the training action vector output by the initialization policy network model and the expected value parameter output by the initialization value network model. Based on the preset discount factor parameter, the target reward value in the target experience tuple, and the expected value parameter, calculate the first supervision signal value corresponding to the initial value network model; Based on the target state vector in the target experience tuple and the training action vector, the training prediction value corresponding to the initial value network model is calculated, and based on the loss value between the training prediction value and the first supervision signal value, the loss value is optimized by backpropagation and gradient descent to update the model parameters in the initial value network model, thereby obtaining the value network model under the training cycle. The value network model outputs the corresponding second supervision signal value based on the training action vector, and the model parameters in the initial policy network model are updated through backpropagation based on the second supervision signal value to obtain the policy network model under the training period. The value network model and its parameters are updated repeatedly over multiple training cycles until the initial seat allocation model shows convergence, thus obtaining the seat allocation model.

8. An optimization device for seat allocation in high-speed trains, characterized in that, The optimization device includes: The request and response module is used to respond to the seat request information sent by the passenger and obtain the seat occupancy information of the target high-speed train at the current time. The matrix construction module is used to construct a seat matrix representing the current occupancy status of the seats of the target high-speed train in each travel section based on the seat occupancy information. The model decision module is used to input the seat matrix and the seat request information into a pre-trained seat allocation model based on Markov decision, so as to use the decision sub-model corresponding to each travel segment in the seat allocation model to make a decision based on the seat request information and the seat matrix, and obtain the local action decision result corresponding to the travel segment output by each decision sub-model. The seat allocation module is used to update the seat matrix based on the local action decision results and the seat request information corresponding to each of the travel segments, so as to obtain an updated seat matrix and determine the seat allocation result corresponding to the seat request information.

9. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. The machine-readable instructions are executed by the processor to perform the steps of the high-speed train seat allocation optimization method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the optimization method for high-speed train seat allocation as described in any one of claims 1 to 7.

Images