Rail transit driving strategy optimization method and system based on federated learning

Abstract

The invention provides a federated learning-based rail transit driving strategy optimization method and system. The method comprises the following steps: after a data user issues a federal learning task, each training node performs reinforcement learning training on a local initial Q network based on respective historical data of a rail transit driving strategy to obtain a respective local Q network decision model; each training node uploads an intermediate gradient parameter of a local Q network decision model of the training node to the block chain network where the training node is located; performing model aggregation operation on each intermediate gradient parameter by each consensus node in the block chain network to obtain a global Q network decision model, and broadcasting the global Q network decision model in the block chain network; and the data user performs rail transit driving strategy optimization based on the global Q network decision model. According to the method, a reasonable optimization strategy can be formulated, the operation efficiency of the urban rail transit system is effectively improved, energy consumption is reduced, and the comfort level of passengers is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of rail transit strategy optimization, and in particular to a rail transit driving strategy optimization method and system based on federated learning. Background Art

[0002] At present, the research on driving strategies of urban rail transit mainly focuses on the use of classical model analysis methods and traditional machine learning methods. By establishing various control models and using a large amount of urban rail transit operation data for training, the system can automatically achieve optimized operation in complex environments and reduce human intervention. Usually, this type of method is server-centric and based on labeled trust data. First, a large-scale training set is established, and then various models are used for classification, clustering, and regression operations to improve urban rail transit driving strategies. In this type of method, the centralized topological structure facilitates the joint learning of multiple rail data operators. The coordinating agency is the central server, which plays the role of coordinating the global model.

[0003] However, while the traditional urban rail transit control method based on machine learning effectively improves driving performance, it also has certain limitations in some aspects. On the one hand, the training of the data set is server-centric, which has high requirements for network bandwidth and communication load. At the same time, the robustness of the training model is relatively general. Once the centralized topology is attacked, the entire training network will fail. On the other hand, due to the particularity of the urban rail transit industry, there are data barriers between upstream and downstream companies, between various departments within the urban rail transit system, and between various urban rail transit operators. Due to the lack of incentive mechanisms, and considering data privacy and information security, the departments that own the data have no intention to open the data, which has a huge impact on promoting data-driven intelligent urban rail transit. Summary of the invention

[0004] The embodiments of the present invention provide a method and system for optimizing rail transit driving strategies based on federated learning to solve the problem that it is difficult to improve the performance of urban rail transit driving strategies.

[0005] In a first aspect, an embodiment of the present invention provides a rail transit driving strategy optimization method based on federated learning, comprising:

[0006] After the data user publishes the federated learning task, each training node performs reinforcement learning training on the local initial Q network based on its own rail transit driving strategy historical data to obtain its own local Q network decision model;

[0007] Each training node uploads the intermediate gradient parameters of its local Q network decision model to the blockchain network where it is located;

[0008] Each consensus node in the blockchain network performs model aggregation operations on each intermediate gradient parameter to obtain the global Q network decision model and broadcast it in the blockchain network;

[0009] Data users optimize rail transit driving strategies based on the global Q-network decision model.

[0010] In one possible implementation, each training node also uploads the training loss of its local Q-network decision model to the blockchain network;

[0011] Each consensus node in the blockchain network performs model aggregation operations on each intermediate gradient parameter, including:

[0012] Each consensus node in the blockchain network performs quality assessment on the local Q network decision model of each training node based on the training loss, and performs model aggregation operation on the intermediate gradient parameters of each local Q network decision model based on the quality assessment results of each local Q network decision model.

[0013] In a possible implementation, the training loss is cross entropy; each consensus node in the blockchain network performs a quality assessment on the local Q network decision model of each training node based on the training loss, and performs a model aggregation operation on the intermediate gradient parameters of each local Q network decision model based on the quality assessment results of each local Q network decision model, including:

[0014] For each training node, if the cross entropy of the training node is greater than a preset threshold, the quality assessment result of the local Q network decision model of the training node is determined to be unqualified, and the quality assessment result is sent to the training node, so that the training node re-performs reinforcement learning training on the local initial Q network based on the respective rail transit driving strategy historical data;

[0015] Based on the quality assessment results of each local Q network decision model whose quality assessment results are qualified, a model aggregation operation is performed on the intermediate gradient parameters of each local Q network decision model.

[0016] In one possible implementation, the formula for performing model aggregation operations is:

[0017]

[0018] Among them, F(m) is the global objective function obtained by model aggregation operation after the mth training, N is the total number of nodes participating in federated learning training, and F k is the local objective function of the kth training node at the mth training, w k is the influence weight of the kth training node, F k(m) is the empirical risk of the k-th training node when training the model based on local data for the mth time.

[0019] In a possible implementation, the federated learning task includes a training life cycle, a model preset performance index, an initialization model, a number of training rounds, and a learning rate; wherein the initialization model uses the train as an intelligent agent, the line condition as an interactive environment, the position and speed of the train as a state, and the unit mass output force as an action;

[0020] The expression of the local Q network decision model is:

[0021] Q(s,a)=E[R t+1 +γmax a Q(s′, a′)|s t =s,a t =a]

[0022] Among them, Q(s, a) is the state action value function of the local Q network decision model, E is the expectation, R t+1 is the reward obtained at time t+1, γ is the discount factor, s′ is the new state at time t+1, and a′ is the possible action to be taken in state s′.

[0023] In a possible implementation, each training node performs reinforcement learning training on the local initial Q network based on its own rail transit driving strategy historical data. The update iteration expression is:

[0024] Q(s t , a t )←Q(s t , a t )+α[R t+1 +γmax a Q(s′, a′)-Q(s t , a t )]

[0025] Among them, α is the learning rate.

[0026] In a possible implementation, the method further includes:

[0027] Each training node repeatedly performs reinforcement learning training on the local initial Q network and uploads the intermediate gradient parameters to the blockchain network. Each consensus node in the blockchain network repeatedly performs model aggregation operations on each intermediate gradient parameter to obtain multiple global Q network decision models until the model preset performance indicators of the federated learning task are met or the training survival time expires;

[0028] The data users verified each global Q-network decision model respectively, and selected the intermediate gradient parameters of the global Q-network decision model with the best effect to optimize the rail transit driving strategy.

[0029] In a second aspect, an embodiment of the present invention provides a rail transit driving strategy optimization system based on federated learning, including:

[0030] Training nodes are used to perform reinforcement learning training on the local initial Q network based on their respective rail transit driving strategy historical data after the data user publishes the federated learning task, obtain their respective local Q network decision models, and upload the intermediate gradient parameters of their respective local Q network decision models to the blockchain network;

[0031] Consensus nodes are used to perform model aggregation operations on various intermediate gradient parameters, obtain the global Q network decision model, and broadcast it in the blockchain network;

[0032] Data users are used to optimize rail transit driving strategies based on the global Q-network decision model.

[0033] In a third aspect, an embodiment of the present invention provides a terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the method described in the first aspect or any possible implementation manner of the first aspect are implemented.

[0034] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described in the first aspect or any possible implementation of the first aspect are implemented.

[0035] The embodiment of the present invention provides a method and system for optimizing rail transit driving strategies based on federated learning. On the basis of fully considering the privacy protection of the original data of each data owner of urban rail transit, by constructing a Q network decision model for train operation adjustment, the operation process of the train under different conditions is simulated, and the optimal train operation adjustment strategy is learned. At the same time, blockchain technology is used to share the driving data of multiple city operators to distribute the training and optimization of the intelligent driving network model, and a federated transfer learning method based on blockchain technology is used to optimize the urban rail transit strategy to improve the overall driving performance. Compared with the traditional urban rail transit strategy scheduling method, the urban rail transit driving strategy optimization method based on the integration of blockchain and federated learning can better adapt to the changes in environmental parameters, formulate reasonable optimization strategies, effectively improve the operating efficiency of the urban rail transit system, and achieve energy consumption reduction and passenger comfort. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0037] Figure 1 This is an application scenario diagram of a rail transit driving strategy optimization method based on federated learning provided by an embodiment of the present invention;

[0038] Figure 2 is a flow chart of an implementation method of a rail transit driving strategy optimization method based on federated learning provided by an embodiment of the present invention;

[0039] Figure 3 This is a schematic diagram of the data interaction relationship between various nodes in a blockchain network provided by an embodiment of the present invention;

[0040] Figure 4 Schematic diagram of the training process of the Q algorithm provided by one embodiment of the present invention;

[0041] Figure 5 is a flow chart of an implementation method of a rail transit driving strategy optimization method based on federated learning provided by another embodiment of the present invention;

[0042] Figure 6 It is a structural schematic diagram of a rail transit driving strategy optimization system based on federated learning provided by an embodiment of the present invention;

[0043] Figure 7 is a schematic diagram of a terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.

[0045] In order to make the purpose, technical solutions and advantages of the present invention more clear, specific embodiments will be described below in conjunction with the accompanying drawings.

[0046] In the context of the big data-driven intelligent era, the field of urban rail transit has ushered in a new round of management technology innovation. The wide application of artificial intelligence technology in urban rail transit information management scenarios such as passenger flow prediction, automatic ticket sales and inspection, passenger information query, and face recognition security inspection has greatly improved the efficiency and quality of urban rail transit management in my country. However, since urban rail transit is a complex giant system, there are significant differences in business attributes, system architecture, security features, and principle processes among various business systems. For the sake of data privacy and security protection, the information interaction frequency between various business systems is low, the granularity is small, and they are independent of each other. Therefore, a large amount of data is still scattered in various urban rail transit operators or various professional departments in the form of data islands. The data barriers that are difficult to break between departments have seriously hindered the development of intelligent urban rail transit. Data exchange and use and data security protection seem to be irreconcilable contradictions. Due to the lack of an effective computing power sharing mechanism and computing trust management mechanism, it is difficult for the system to optimize the driving strategy of urban rail transit, especially to cope with sudden large passenger flows caused by large gatherings and emergency events, and it is impossible to quickly evacuate passengers, which reduces the quality of urban rail transit services and easily causes safety accidents. Therefore, how to effectively use these data to optimize urban rail transit driving strategies while ensuring data security and privacy has become an urgent problem to be solved.

[0047] At present, there are various methods for optimizing the driving strategy of urban rail transit, and each method has its own advantages and disadvantages and applicable scenarios. In practical applications, it is necessary to select a suitable algorithm according to the characteristics and requirements of the specific problem, and make corresponding improvements and optimizations to the problems existing in the algorithm.

[0048] Genetic algorithm: It simulates the natural selection and inheritance mechanism in the biological evolution process, and continuously iterates and optimizes the solution set through operations such as selection, crossover (hybridization) and mutation. In the optimization of urban rail transit driving strategies, genetic algorithms usually encode different driving strategies into chromosomes, evaluate the pros and cons of chromosomes through fitness functions, and perform selection and genetic operations accordingly to find the optimal or approximately optimal driving strategy. However, when solving complex problems, genetic algorithms may face the problems of large amount of calculation and slow convergence. The algorithm's parameter settings (such as population size, crossover probability, mutation probability, etc.) have a great influence on the solution results and need to be carefully adjusted.

[0049] Particle swarm optimization algorithm: It simulates the social behavior of bird flocks foraging and finds the optimal solution to the problem through information sharing and collaboration between particles. In the optimization of urban rail transit driving strategies, each particle represents a potential driving strategy, and the particle approaches the optimal solution by updating its own speed and position. However, when solving complex problems, the particle swarm optimization algorithm is prone to fall into the local optimal solution. The performance of the algorithm is greatly affected by parameters such as the number of particles and the learning factor.

[0050] Reinforcement learning algorithm: Through interactive learning between the agent and the environment, the driving strategy is continuously adjusted to maximize a certain reward (such as reducing energy consumption, improving punctuality, etc.). However, it requires a large amount of training data and computing resources, and the model has poor interpretability, requiring a long training time and high computing cost.

[0051] Multi-objective optimization algorithm: By introducing objective weights and multi-dimensional optimization strategies, multiple performance indicators (such as energy consumption, punctuality, comfort, safety, etc.) are comprehensively considered to find the optimal driving strategy. However, how to reasonably set the objective weights to balance the conflicts between different objectives is an issue that requires careful consideration. The computational complexity of multi-objective optimization algorithms is usually high and requires a long solution time.

[0052] As a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, and encryption algorithm, blockchain technology is conducive to combining the distributed characteristics of urban rail transit data and the advantages of federated learning, and establishing a mutually trusted big data sharing and machine learning framework under the premise of meeting data privacy, security, and regulatory requirements. Blockchain technology, with its decentralized and tamper-proof characteristics, provides a more secure and transparent environment for federated learning. In the distributed federated learning of urban rail transit based on blockchain, blockchain is mainly used to record historical information of model updates, verify the identity and contribution of participants, and implement fair incentive mechanisms. Specifically, blockchain can ensure the authenticity and integrity of model updates and prevent data from being tampered with or forged; at the same time, through technical means such as smart contracts, incentive mechanisms can be automatically executed, and rewards can be distributed according to the contribution of participants, thereby stimulating the enthusiasm of participants and better realizing safe and reliable computing power sharing.

[0053] As a new type of artificial intelligence basic technology for data silos and privacy protection, federated learning was first proposed by Google. It is a distributed machine learning algorithm. Its design goal is to carry out efficient machine learning between multiple participants or multiple computing nodes under the premise of ensuring information security during big data exchange, protecting terminal data and personal data privacy, and legal compliance. It allows models to be trained on local devices of multiple participants (such as different urban rail transit systems) without centralizing the original data to a central server. Federated learning can achieve that the own data of each enterprise or department does not leave the local area, but through the parameter exchange method under the encryption mechanism, collaborative training and co-construction of models are carried out, and a virtual shared model is established without violating data privacy regulations. This method can effectively solve the problem of data privacy protection, because the participants only exchange model update information (such as weights) instead of directly sharing the original data. In the field of urban rail transit, federated learning can be used to optimize tasks such as train scheduling and fault prediction, and improve the overall operational efficiency of the system.

[0054] Blockchain and federated learning technologies are highly complementary. In the intelligent driving scenario, blockchain can effectively enhance the security, privacy and credibility of federated learning. In the process of federated learning, each edge computing node has a large amount of historical data. As a distributed machine learning technology, federated learning allows multiple participants to achieve joint modeling by exchanging encrypted machine learning intermediate results without disclosing the underlying data. This technical feature allows data to be processed locally without being centralized on a centralized server, thereby protecting data privacy. Blockchain technology is a decentralized distributed ledger that ensures the security and reliability of data among multiple untrusted distributed participants through technical means such as consensus mechanisms and smart contracts. The distributed characteristics of blockchain are consistent with the distributed machine learning concept of federated learning, and together promote the decentralized processing and storage of data; in the model calculation process, federated learning exchanges encrypted machine learning intermediate results, and federated learning can achieve joint modeling without leaking the original data, thereby protecting the privacy and security of the data. Blockchain technology ensures the transparency, traceability and tamper-proofness of on-chain data through technical means such as asymmetric encryption and hash algorithms. This technical feature provides additional security for federated learning, making data exchange in the joint modeling process more secure and reliable; at the same time, federated learning can also make full use of blockchain authentication and consensus mechanisms to enhance its own security and credibility. For example, storing the model parameters of federated learning in the blockchain can ensure the security and reliability of the model parameters; at the same time, the consensus mechanism of the blockchain can further reduce the risk of data tampering.

[0055] Based on the above problems, in order to solve the problems that the existing mainstream urban rail transit control strategy optimization algorithms use model analysis and deep learning algorithms, such as the inability to adapt to complex environments, data islands, and centralization, the present invention provides an urban rail transit driving strategy optimization method based on the integration of blockchain and federated learning. It fully considers the original data protection of each data owner of urban rail transit, uses historical operation data, and adopts a data-driven approach to share model weights through a decentralized blockchain network, so that each data generator does not need to share data through a third party, but only needs to perform supervised learning model training on their respective computing servers. Each data owner uses the training weights of the previous data owner obtained from the blockchain as the starting point of training. The batch gradient update algorithm is used to update the parameters. By minimizing the loss function, the update of the local network model and the fusion network parameters is completed, and finally a more efficient urban rail transit driving strategy model is obtained.

[0056] Aiming at the application scenarios of urban rail transit and based on the problem of train operation adjustment, the present invention proposes an urban rail transit driving strategy optimization method based on the integration of blockchain and federated learning. On the basis of fully considering the privacy protection of the original data of each data owner of urban rail transit, a decision model for train operation adjustment is constructed to simulate the operation process of the train under different states, and the optimal train operation adjustment strategy is learned.

[0057] Figure 1 This is an application scenario diagram of the rail transit driving strategy optimization method based on federated learning provided in an embodiment of the present invention. Figure 1 As shown, in order to make full use of the driving data of multiple city operators to distribute and optimize the intelligent driving network model, the present invention proposes a federated transfer learning Q algorithm based on the blockchain system, which overcomes the defects of the traditional model parsing algorithm. The embodiment of the present invention uses the blockchain as a trusted model parameter sharing platform, records the intermediate gradient parameters and training losses of local training, and implements the federated learning model aggregation through the execution of smart contracts by blockchain nodes, reducing the malicious attacks and untrustworthy risks of centralized parameter servers. Specifically, each data owner participating in federated learning uses its local data server as a blockchain node to obtain the Q network parameters protected by differential privacy from the blockchain. The update of the parameters adopts a batch gradient update algorithm, and updates the local Q network and fusion network parameters by minimizing the Q value loss function, and finally obtains a more efficient urban rail transit train control model. Different from the general Q algorithm, the present invention makes full use of the decentralized characteristics of the blockchain to construct a distributed trusted computing framework for federated learning, and realizes the fairness of participating nodes and trusted automated model aggregation through incentive mechanisms and smart contracts, solving the problem that rail data operators may do evil or be maliciously attacked during the process of participating in federated learning.

[0058] See also Figure 2 , which shows a flow chart of the implementation of the rail transit driving strategy optimization method based on federated learning provided by an embodiment of the present invention, which is described in detail as follows:

[0059] Step 201, after the data user publishes the federated learning task, each training node performs reinforcement learning training on the local initial Q network based on its own rail transit driving strategy historical data to obtain its own local Q network decision model.

[0060] In this embodiment, the present invention uses blockchain as a trusted model parameter sharing platform to record the intermediate gradient parameters and training losses of local training of each data owner, and realizes federated learning model aggregation by executing smart contracts through blockchain nodes.

[0061] First, a decentralized peer-to-peer network is established between the nodes of urban rail transit operators. Federated learning obtains a unified global model through cooperative training of local data of participating nodes. Among them, the participating rail data operator nodes represent the data owners of the model and have complete autonomy over local data. The parameter server is responsible for aggregating the intermediate parameters of the training nodes to form a global model.

[0062] Before starting training, the publisher of the federated learning task needs to establish a joint training group. First, each training node declares the data information it has in advance and aggregates local models with strong correlations. Secondly, in the process of blockchain executing the model aggregation algorithm, each node uploads the intermediate gradient parameters to the consensus node in the blockchain network after local training for model aggregation calculation. Each node competes for the right to record and records the consensus aggregate model results on the chain. After the participating training nodes obtain the local model, they broadcast it to the collaborative nodes in the joint training group for model quality verification. The collaborative nodes use their own local data to verify the model, and broadcast the verification results after adding data signatures. Finally, the model parameters after verification form the current round of aggregation model through the adaptive model aggregation algorithm, which is returned by the master node in the blockchain network to the participating nodes in the joint training group to start the next round of model update iteration.

[0063] In step 202, each training node uploads the intermediate gradient parameters of its local Q network decision model to the blockchain network where it is located.

[0064] In this embodiment, due to data volume and privacy considerations, the local data of participating nodes cannot be directly stored on the blockchain. When a data provider participates in the federated learning process, a unique distributed identity will be generated and stored on the chain. At the same time, the data configuration file parameters will be saved, including the keywords, type, size, hash, etc. of the data. The configuration file parameters are stored on the blockchain in the form of key-value pairs of the contract account.

[0065] In step 203, each consensus node in the blockchain network performs model aggregation operations on each intermediate gradient parameter to obtain a global Q network decision model, and broadcasts it in the blockchain network.

[0066] In this embodiment, after each training node completes the training of the local model, it enters the global model verification stage. Each consensus node performs quality assessment on the local models uploaded by each training node and aggregates them into a global model. Competitive accounting saves the aggregated global model on the chain.

[0067] At this time, if the completion conditions of the federated learning task published by the data user are met, the aggregated global model can be broadcast in the blockchain network, so that the data user can obtain the final global Q network decision model; if the completion conditions are not met, for example, the training rounds do not meet the requirements or the model indicators do not meet the requirements, then after the broadcast, each training node will use the aggregated model as the basis for the next round of training and enter the next round of training.

[0068] In step 204 , the data user optimizes the rail transit driving strategy based on the global Q network decision model.

[0069] In this embodiment, through the above steps, data users can obtain a global Q network decision model that meets the federated learning tasks they publish, use blockchain technology to share driving data of multiple city operators to distribute training and optimize the intelligent driving network model, and use a federated transfer learning method based on blockchain technology to optimize urban rail transit strategies and improve overall driving performance. Compared with traditional urban rail transit strategy scheduling methods, the urban rail transit driving strategy optimization method based on the integration of blockchain and federated learning can better adapt to changes in environmental parameters, formulate reasonable optimization strategies, effectively improve the operating efficiency of urban rail transit systems, reduce energy consumption, and improve passenger comfort.

[0070] The embodiment of the present invention, on the basis of fully considering the privacy protection of the original data of each data owner of urban rail transit, constructs a Q network decision model for train operation adjustment, simulates the operation process of the train under different conditions, and learns the optimal train operation adjustment strategy. At the same time, blockchain technology is used to share the driving data of multiple city operators to distribute the training and optimization of the intelligent driving network model, and a federated transfer learning method based on blockchain technology is used to optimize the urban rail transit strategy to improve the overall driving performance. Compared with the traditional urban rail transit strategy scheduling method, the urban rail transit driving strategy optimization method based on the integration of blockchain and federated learning can better adapt to the changes in environmental parameters, formulate reasonable optimization strategies, effectively improve the operating efficiency of the urban rail transit system, and achieve energy reduction and passenger comfort.

[0071] In one possible implementation, each training node also uploads the training loss of its local Q-network decision model to the blockchain network;

[0072] Each consensus node in the blockchain network performs model aggregation operations on each intermediate gradient parameter, including:

[0073] Each consensus node in the blockchain network performs quality assessment on the local Q network decision model of each training node based on the training loss, and performs model aggregation operation on the intermediate gradient parameters of each local Q network decision model based on the quality assessment results of each local Q network decision model.

[0074] In this embodiment, the federated learning training process needs to jointly model the data provided by multiple urban rail data operators, and the amount of information in the data determines the accuracy of the overall aggregation model. The present invention uses data cross entropy to measure the amount of information contained in the data set, and uses it as a basis for judging the contribution of data providers, thereby raising the entry threshold for federated learning joint modeling, preventing the introduction of low-quality models from the source, and serving as a basis for fair distribution of model benefits.

[0075] In a possible implementation, the training loss is cross entropy; each consensus node in the blockchain network performs a quality assessment on the local Q network decision model of each training node based on the training loss, and performs a model aggregation operation on the intermediate gradient parameters of each local Q network decision model based on the quality assessment results of each local Q network decision model, including:

[0076] For each training node, if the cross entropy of the training node is greater than a preset threshold, the quality assessment result of the local Q network decision model of the training node is determined to be unqualified, and the quality assessment result is sent to the training node, so that the training node re-performs reinforcement learning training on the local initial Q network based on the respective rail transit driving strategy historical data;

[0077] Based on the quality assessment results of each local Q network decision model whose quality assessment results are qualified, a model aggregation operation is performed on the intermediate gradient parameters of each local Q network decision model.

[0078] In this embodiment, the quality assessment of the local model is mainly based on the cross entropy in the local training process, which is used to measure the deviation between the model output and the true result. It is assumed that all participating nodes in the federated learning have a labeled dataset D t ={(x1,y1),(x2,y2),......(x n ,y n )} is used to verify the quality of the model. The cross entropy can be calculated by the following formula:

[0079]

[0080] where y i Indicates the value of the label data, f(x i ) represents the prediction result of the model. The smaller the value of cross entropy, the closer the probability distribution predicted by the model is to the true result, and the higher the quality of model training.

[0081] In one possible implementation, the formula for performing model aggregation operations is:

[0082]

[0083] Among them, F(m) is the global objective function obtained by model aggregation operation after the mth training, N is the total number of nodes participating in federated learning training, and F k is the local objective function of the kth training node at the mth training, w k is the influence weight of the kth training node, F k (m) is the empirical risk of the k-th training node when training the model based on local data for the mth time.

[0084] In this embodiment, the distributed federated reinforcement learning training process proposed for the urban rail transit driving strategy is mainly divided into two parts: local model training of data holders and global model verification of data users. The data fusion process involved is as follows: Figure 3 shown.

[0085] During the local model training process, the data holder uses the local data (s, a, s', r) to broadcast the model w to the data user after the differential privacy mechanism is protected. i Train and obtain the network parameter gradient value dθ, which is then encrypted asymmetrically using public_key(Q h ) and data signature hash (Q h ) and then aggregated to data users through the blockchain network. The expression is as follows:

[0086]

[0087] During the global model verification process, data users use the model gradients received within the cycle to update local model parameters, and verify the model effects separately. They select the model parameters with the best model effects during multiple training processes, encrypt them with the differential privacy protection mechanism, and broadcast them to each participating node. Then each node continues training based on this model.

[0088]

[0089] After the training nodes in the blockchain use local data to train local models, the current mainstream in the industry is to use the federated average method to aggregate the local models, but this method does not take into account the problem of reduced global model quality caused by the participation of low-quality models in aggregation. In the embodiment of the present invention, an adaptive model aggregation algorithm based on cross entropy is adopted. First, unqualified local models are eliminated by the value of cross entropy, and the remaining local models are used for aggregation. Then, the cross entropy of each training node is used to evaluate the model quality, and it is used to adjust the corresponding local model aggregation weight, and the accuracy of the aggregated model is further improved by improving the contribution of high-quality model parameters in the aggregated model.

[0090] The evaluation of model quality mainly depends on the node p k The model loss of the local training in the i-th round is used to determine the cross entropy H(F(x i ,y i )) to evaluate the local model loss, node p k The quality assessment weight w i (p k ) can be defined as:

[0091]

[0092] Among them, N is the total number of training rounds, H k (f(x i ,y i )) is the node p k The cross entropy of the i-th round of training.

[0093] In addition, in order to avoid participating blockchain nodes from providing false model quality evaluation results, the present invention adopts an asynchronous parameter audit mechanism for parameter verification. k Broadcast its own local model parameter m in the blockchain network k , and the cross entropy H(f(x i ,y i ), other participating nodes, i.e., collaborative nodes, use their own locally labeled dataset D t ={(x1, y1), (x2, y2),...(x n ,y n )}Verification node p k The quality of the local model is determined by the node’s self-interest.

[0094] In a possible implementation, the federated learning task includes a training life cycle, a model preset performance index, an initialization model, a number of training rounds, and a learning rate; wherein the initialization model uses the train as an intelligent agent, the line condition as an interactive environment, the position and speed of the train as a state, and the unit mass output force as an action;

[0095] The expression of the local Q network decision model is:

[0096] Q(s,a)=E[R t+1 +γmax a Q(s′, a′)|s t =s,a t =a]

[0097] Among them, Q(s, a) is the state action value function of the local Q network decision model, E is the expectation, R t+1 is the reward obtained at time t+1, γ is the discount factor, s′ is the new state at time t+1, and a′ is the possible action to be taken in state s′.

[0098] In this embodiment, the urban rail transit line has a high density and repetition rate. In the communication-based train automatic control system, the automatic train driving system uses on-board and ground information to control the train traction, braking, coasting / cruising, etc. under the supervision of the train automatic protection system. In order to realize intelligent train driving, it is necessary to calculate the optimal driving control strategy. The present invention uses the federated transfer learning Q algorithm to model the urban rail transit driving strategy. On the basis of the Markov decision process, the train is used as an intelligent agent and the line conditions are used as the interactive environment. The optimal control strategy π is calculated through a deep reinforcement learning algorithm based on the Q network. * (a|s)=P[A t =a|S t =s].

[0099] First, the control process during train operation is redefined as a Markov decision process (MDP) quaternion {S, A, P, R, γ}, where:

[0100] S represents the running state of the intelligent train, which represents the current position and speed of the train;

[0101] A represents the action of the agent, which represents the unit mass output force of the train;

[0102] P represents the probability of state transition distribution. Since the strategy and state transition are random, the trajectory obtained by simulation is random.

[0103] R represents the reward obtained by the interaction between the agent and the environment, which represents the overall performance index of the train operation;

[0104] γ represents the discount factor, which is used to solve the uncertainty problem of future rewards. In actual modeling, unless the MDP with infinite time steps has convergence behavior, it is inappropriate to consider the rewards at infinity, which is not conducive to solving the MDP. For this reason, a discount mechanism is introduced to obtain discounted rewards.

[0105] The structure of the Q algorithm is as follows Figure 4 As shown in the figure, it is mainly divided into three parts. The Q network generates the Q value under the current state and behavior. The target Q network mainly generates the optimal target Q value for the next state. The output of the target Q network plus the current reward R, minus the output of the Q network, will be used as the gradient loss to update the weights of the Q network. The state transfer memory will store the state transfer process of the train, and the randomly selected samples will be used to train the Q network.

[0106] In a possible implementation, each training node performs reinforcement learning training on the local initial Q network based on its own rail transit driving strategy historical data. The update iteration expression is:

[0107] Q(s t , a t )←Q(s t , a t )+α[R t+1 +γmax a Q(s′, a′)-Q(s t , a t )]

[0108] Among them, α is the learning rate.

[0109] In this embodiment, after setting the variables according to the urban rail transit application scenario, the function Q(s, a) can be defined as the expected reward obtained after taking action a from state s and following strategy π, and the expression is as follows:

[0110] Q(s,a)=E[R t+1 +γmax a Q(s′, a′)|s t =s,a t =a]

[0111] Among them, R t+1 is the reward obtained at time t+1, S′ is the new state at time t+1, and a′ is the possible action to be taken in the new state. After defining the quaternary group of urban rail transit during operation as the parameters of the Q algorithm, the algorithm is executed until the training survival time expires or the preset value of the model effect is met. The Q value update iteration expression is as follows, where α is the learning rate, which indicates the speed at which new information covers old information:

[0112] Q(s t , a t )←Q(s t , a t )+α[R t+1 +γmax a Q(s′, a′)-Q(s t , a t )]

[0113] In a possible implementation, the method further includes:

[0114] Each training node repeatedly performs reinforcement learning training on the local initial Q network and uploads the intermediate gradient parameters to the blockchain network. Each consensus node in the blockchain network repeatedly performs model aggregation operations on each intermediate gradient parameter to obtain multiple global Q network decision models until the model preset performance indicators of the federated learning task are met or the training survival time expires;

[0115] The data users verified each global Q-network decision model respectively, and selected the intermediate gradient parameters of the global Q-network decision model with the best effect to optimize the rail transit driving strategy.

[0116] In this embodiment, local model training and global model verification are repeated until the preset performance indicators of the model are met or the training survival time expires. The node that finally provides the optimal model adds the next block to record the model information and transaction information, and obtains corresponding rewards. The use of blockchain technology can protect the data security of data holders, encourage more urban rail transit operator nodes to participate in model co-construction, and automatically ignore malicious nodes that provide poor quality data. The data privacy of each operator can be protected through intelligent driving federated learning based on distributed blockchain, and operators in each city can obtain more efficient, energy-saving and safe intelligent driving strategies by accessing this system training.

[0117] In a specific embodiment, the complete process of the rail transit driving strategy optimization method based on federated learning is as follows: Figure 5 As shown, the following steps are included:

[0118] ① Blockchain initialization. After the data user publishes the federated learning task, the nodes of each city rail transit operator rely on blockchain technology to establish a decentralized peer-to-peer network and federated learning environment. The design of the blockchain includes three main parts: block header information, block personalized information, and block verification information. The block header mainly records some information required for the historical records and traceability of the blockchain; the block personalized information mainly contains the information and attributes required for the federated learning training process; the block verification information is mainly the information used by the blockchain itself to verify the accuracy of its content. By recording the hash value of the parent block and the hash value of the current block, it can be ensured that attackers have no way to destroy the entire blockchain by modifying a single block, ensuring the immutability of the blockchain.

[0119] ② The edge server sends the initialization model, sets the threshold, and publishes the blockchain smart contract, broadcasting hyperparameters such as the training life cycle, model preset performance indicators, initialization model, number of training rounds, and learning rate to all participating nodes on the network;

[0120] ③ Blockchain nodes use local data to train local models. During the training process, each node obtains the network parameter gradient value while protecting the privacy of its own data. At the same time, the trained local model parameters are digitally signed and encrypted with the public key of the data user, and then returned to the edge server of the blockchain system through the peer-to-peer network;

[0121] ④ The blockchain edge server collects the parameters of each participating training node and aggregates the model algorithm, and calculates the contribution of the nodes participating in each model aggregation. Data users use the model gradients received during the cycle to update local model parameters, and verify the model effects respectively, select the model parameters with the best model effect, encrypt them through the differential privacy protection mechanism, and broadcast them to each participating node, and then each node continues training based on this model;

[0122] ⑤ The blockchain node publishes an update block, which contains default information and the aggregated parameters of each participating computing node. In addition, the blockchain also contains information such as the hash value of the previous block, the Merkle tree root, and the hash value of the parameters;

[0123] Repeat steps ② to ⑤. In each round of training, the urban rail operator will use new training parameters for local training and return the training results to the edge server of the blockchain system. After global model aggregation, a new model will be released. This process is repeated until the number of training rounds reaches the preset value, and the optimal strategy is finally obtained.

[0124] As can be seen from the above, compared with the existing track data processing method, the present invention has the following advantages:

[0125] First, the model-free federated transfer learning Q-learning algorithm is adopted. Compared with complex model analysis algorithms, the Q algorithm is more robust to environmental changes and can continue to learn the optimal strategy even if the environment changes.

[0126] Second, the traditional urban rail transit control method based on machine learning is usually server-centric when training data sets, and has high requirements for network bandwidth and communication load. The urban rail transit driving strategy optimization method based on the integration of blockchain and federated learning allows multiple devices or urban rail transit operator entities to conduct distributed learning without the need to centralize data to a central server, thus greatly saving the time and cost of data transmission and storage. At the same time, the use of a decentralized topology can eliminate the dependence of traditional traffic control learning methods on centralized nodes, reduce the risk of single point failures, and effectively improve the robustness and reliability of the system.

[0127] Third, it protects the data privacy of urban rail data operators. The federated learning method allows urban rail transit operators to conduct collaborative training and co-build models through parameter exchange under an encrypted mechanism while retaining their own data privacy. Only model parameters need to be exchanged, which improves data security and effectively learns and analyzes the data set. By updating the parameters of the aggregation server, participants can benefit from the knowledge of other participants and improve the performance of the overall model.

[0128] Fourth, the decentralized federated learning architecture based on blockchain can be applied to a variety of different devices and application scenarios. Participants can also be personal mobile devices, edge computing nodes, etc. The solution has good flexibility and portability.

[0129] In summary, the urban rail transit driving strategy optimization method based on the integration of blockchain and federated learning shows obvious advantages and better results in distributed learning, privacy protection, robustness, etc. These advantages make it a unique and attractive solution for solving driving strategy learning tasks in the field of urban rail transit.

[0130] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0131] The following is a system embodiment of the present invention. For details not described in detail therein, reference may be made to the corresponding method embodiment described above.

[0132] Figure 6 The structural diagram of the rail transit driving strategy optimization system based on federated learning provided by an embodiment of the present invention is shown. For the convenience of explanation, only the part related to the embodiment of the present invention is shown, which is described in detail as follows:

[0133] like Figure 6 As shown, the rail transit driving strategy optimization system 6 based on federated learning includes:

[0134] The training node 61 is used to perform reinforcement learning training on the local initial Q network based on the respective historical data of rail transit driving strategies after the data user 63 releases the federated learning task, obtain the respective local Q network decision model, and upload the intermediate gradient parameters of the respective local Q network decision model to the blockchain network;

[0135] Consensus node 62 is used to perform model aggregation operations on each intermediate gradient parameter to obtain a global Q network decision model and broadcast it in the blockchain network;

[0136] Data user 63 is used to optimize rail transit driving strategy based on the global Q network decision model.

[0137] In a possible implementation, each training node 61 is also used to upload the training loss of the respective local Q network decision model to the blockchain network;

[0138] Each consensus node 62 in the blockchain network is specifically used for:

[0139] The quality of the local Q network decision model of each training node 61 is evaluated based on the training loss, and the model aggregation operation is performed on the intermediate gradient parameters of each local Q network decision model based on the quality evaluation result of each local Q network decision model.

[0140] In a possible implementation, the training loss is cross entropy; each consensus node 62 in the blockchain network is specifically used to:

[0141] For each training node 61, if the cross entropy of the training node 61 is greater than a preset threshold, the quality assessment result of the local Q network decision model of the training node 61 is determined to be unqualified, and the quality assessment result is sent to the training node 61, so that the training node 61 re-performs reinforcement learning training on the local initial Q network based on the respective rail transit driving strategy historical data;

[0142] Based on the quality assessment results of each local Q network decision model whose quality assessment results are qualified, a model aggregation operation is performed on the intermediate gradient parameters of each local Q network decision model.

[0143] In one possible implementation, the formula for performing model aggregation operations is:

[0144]

[0145] Among them, F(m) is the global objective function obtained by model aggregation operation after the mth training, N is the total number of nodes participating in federated learning training, and F k is the local objective function of the kth training node at the mth training, w k is the influence weight of the kth training node, F k (m) is the empirical risk of the k-th training node when training the model based on local data for the mth time.

[0146] In a possible implementation, the federated learning task includes a training life cycle, a model preset performance index, an initialization model, a number of training rounds, and a learning rate; wherein the initialization model uses the train as an intelligent agent, the line condition as an interactive environment, the position and speed of the train as a state, and the unit mass output force as an action;

[0147] The expression of the local Q network decision model is:

[0148] Q(s,a)=E[R t+1 +γmax a Q(s′, a′)|s t =s,a t =a]

[0149] Among them, Q(s, a) is the state action value function of the local Q network decision model, E is the expectation, R t+1 is the reward obtained at time t+1, γ is the discount factor, s′ is the new state at time t+1, and a′ is the possible action to be taken in state s′.

[0150] In a possible implementation, each training node 61 performs reinforcement learning training on the local initial Q network based on its own rail transit driving strategy history data. The update iteration expression is:

[0151] Q(s t , a t )←Q(s t , a t )+α[R t+1 +γmax a Q(s′, a′)-Q(s t , a t )]

[0152] Among them, α is the learning rate.

[0153] In a possible implementation, the method further includes:

[0154] Each training node 61 repeatedly performs reinforcement learning training on the local initial Q network and uploads the intermediate gradient parameters to the blockchain network. Each consensus node 62 in the blockchain network repeatedly performs model aggregation operations on each intermediate gradient parameter to obtain multiple global Q network decision models until the model preset performance indicators of the federated learning task are met or the training survival time expires;

[0155] Data user 63 verifies each global Q network decision model respectively, and selects the intermediate gradient parameters of the global Q network decision model with the best effect to optimize the rail transit driving strategy.

[0156] The embodiment of the present invention, on the basis of fully considering the privacy protection of the original data of each data owner of urban rail transit, constructs a Q network decision model for train operation adjustment, simulates the operation process of the train under different conditions, and learns the optimal train operation adjustment strategy. At the same time, blockchain technology is used to share the driving data of multiple city operators to distribute the training and optimization of the intelligent driving network model, and a federated transfer learning method based on blockchain technology is used to optimize the urban rail transit strategy to improve the overall driving performance. Compared with the traditional urban rail transit strategy scheduling method, the urban rail transit driving strategy optimization method based on the integration of blockchain and federated learning can better adapt to the changes in environmental parameters, formulate reasonable optimization strategies, effectively improve the operating efficiency of the urban rail transit system, and achieve energy reduction and passenger comfort.

[0157] Figure 7 is a schematic diagram of a terminal provided by an embodiment of the present invention. Figure 7 As shown, the terminal 7 of this embodiment includes: a processor 70, a memory 71, and a computer program 72 stored in the memory 71 and executable on the processor 70. When the processor 70 executes the computer program 72, the steps in the above-mentioned embodiments of the rail transit driving strategy optimization method based on federated learning are implemented, for example Figure 2 Alternatively, when the processor 70 executes the computer program 72, the functions of each part in the above-mentioned system embodiments are realized, for example, Figure 6 Functions of modules / units 61 to 663 shown.

[0158] Exemplarily, the computer program 72 may be divided into one or more modules / units, which are stored in the memory 71 and executed by the processor 70 to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program 72 in the terminal 7. For example, the computer program 72 may be divided into Figure 6 Modules / units 61 to 63 are shown.

[0159] The terminal 7 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The terminal 7 may include, but is not limited to, a processor 70 and a memory 71. Those skilled in the art will appreciate that Figure 7 It is only an example of terminal 7 and does not constitute a limitation on terminal 7. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal may also include input and output devices, network access devices, buses, etc.

[0160] The processor 70 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0161] The memory 71 may be an internal storage unit of the terminal 7, such as a hard disk or memory of the terminal 7. The memory 71 may also be an external storage device of the terminal 7, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal 7. Further, the memory 71 may also include both an internal storage unit of the terminal 7 and an external storage device. The memory 71 is used to store the computer program and other programs and data required by the terminal. The memory 71 may also be used to temporarily store data that has been output or is to be output.

[0162] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0163] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0164] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0165] In the embodiments provided by the present invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0166] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0167] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0168] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned rail transit driving strategy optimization method embodiments based on federated learning can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device that can carry the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practices in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practices, computer-readable media does not include electrical carrier signals and telecommunication signals.

[0169] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A rail transit driving strategy optimization method based on federated learning, characterized in that: include: After the data user publishes the federated learning task, each training node performs reinforcement learning training on the local initial Q network based on its own rail transit driving strategy historical data to obtain its own local Q network decision model; Each training node uploads the intermediate gradient parameters of its local Q network decision model to the blockchain network where it is located; Each consensus node in the blockchain network performs model aggregation operations on each intermediate gradient parameter to obtain a global Q network decision model, and broadcasts it in the blockchain network; The data user optimizes rail transit driving strategy based on the global Q network decision model.

2. The rail transit driving strategy optimization method based on federated learning according to claim 1 is characterized in that: Each training node also uploads the training loss of its respective local Q network decision model to the blockchain network; Each consensus node in the blockchain network performs model aggregation operations on each intermediate gradient parameter, including: Each consensus node in the blockchain network performs quality assessment on the local Q network decision model of each training node based on the training loss, and performs model aggregation operation on the intermediate gradient parameters of each local Q network decision model based on the quality assessment results of each local Q network decision model.

3. The rail transit driving strategy optimization method based on federated learning according to claim 2 is characterized in that: The training loss is cross entropy; each consensus node in the blockchain network performs a quality assessment on the local Q network decision model of each training node based on the training loss, and performs a model aggregation operation on the intermediate gradient parameters of each local Q network decision model based on the quality assessment results of each local Q network decision model, including: For each training node, if the cross entropy of the training node is greater than a preset threshold, the quality assessment result of the local Q network decision model of the training node is determined to be unqualified, and the quality assessment result is sent to the training node, so that the training node re-performs reinforcement learning training on the local initial Q network based on the respective rail transit driving strategy historical data; Based on the quality assessment results of each local Q network decision model whose quality assessment results are qualified, a model aggregation operation is performed on the intermediate gradient parameters of each local Q network decision model.

4. The rail transit driving strategy optimization method based on federated learning according to claim 2 is characterized in that: The formula for model aggregation operation is: Among them, F(m) is the global objective function obtained by model aggregation operation after the mth training, N is the total number of nodes participating in federated learning training, and F k is the local objective function of the kth training node at the mth training, w k is the influence weight of the kth training node, F k (m) is the empirical risk of the k-th training node when training the model based on local data for the mth time.

5. The rail transit driving strategy optimization method based on federated learning according to claim 1 is characterized in that: The federated learning task includes a training life cycle, a model preset performance index, an initialization model, a number of training rounds, and a learning rate; wherein the initialization model takes the train as an intelligent agent, the line condition as an interactive environment, the position and speed of the train as a state, and the unit mass output force as an action; The expression of the local Q network decision model is: Q(s,a)=E[R t+1 +γmax a Q(s′,a′)|s t =s,a t =a] Among them, Q(s, a) is the state action value function of the local Q network decision model, E is the expectation, R t+1 is the reward obtained at time t+1, γ is the discount factor, s′ is the new state at time t+1, and a′ is the possible action to be taken in state s′.

6. The rail transit driving strategy optimization method based on federated learning according to claim 5 is characterized in that: The update iterative expression of each training node performing reinforcement learning training on the local initial Q network based on its own rail transit driving strategy historical data is: Q(s t ,a t )←Q(s t ,a t )+α[R t+1 +γmax a Q(s′,a′)-Q(s t ,a t )] Among them, α is the learning rate.

7. The rail transit driving strategy optimization method based on federated learning according to claim 5 is characterized in that: Also includes: Each training node repeatedly performs reinforcement learning training on the local initial Q network and uploads the intermediate gradient parameters to the blockchain network. Each consensus node in the blockchain network repeatedly performs model aggregation operations on each intermediate gradient parameter to obtain multiple global Q network decision models until the model preset performance index of the federated learning task is met or the training survival time expires; The data user verifies each global Q network decision model respectively, and selects the intermediate gradient parameters of the global Q network decision model with the best effect to optimize the rail transit driving strategy.

8. A rail transit driving strategy optimization system based on federated learning, characterized in that: include: Training nodes are used to perform reinforcement learning training on the local initial Q network based on their respective rail transit driving strategy historical data after the data user publishes the federated learning task, obtain their respective local Q network decision models, and upload the intermediate gradient parameters of their respective local Q network decision models to the blockchain network; The consensus node is used to perform model aggregation operations on various intermediate gradient parameters to obtain a global Q network decision model and broadcast it in the blockchain network; The data user is used to optimize the rail transit driving strategy based on the global Q network decision model.

9. A terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method as claimed in any one of claims 1 to 7 are implemented.

Images