Intelligent traffic system control method and system based on type-2 fuzzy and reinforcement learning

Abstract

The invention discloses an intelligent traffic system control method and system based on type-2 fuzzy and reinforcement learning, and belongs to the technical field of intelligent traffic control, and the control method comprises the steps: employing a type-2 fuzzy logic model to process collected real-time data of a traffic system, and outputting a control signal of a traffic signal lamp; and outputting a traffic flow prediction result by using the established traffic flow prediction model. Judging whether the control strategy of the current traffic signal lamp is effective or not, if so, optimizing the control strategy of the traffic signal lamp by using a DQN algorithm, and if not, optimizing the control strategy of the traffic signal lamp by using an FDQN algorithm; adjusting the control signal of the traffic signal lamp based on the traffic flow prediction result and the optimized control strategy of the traffic signal lamp; according to the method, the type-2 fuzzy logic system and the deep reinforcement learning algorithm are fused, the uncertainty in the traffic flow can be effectively processed, and the traffic signal control strategy is optimized in real time, so that the overall efficiency of the traffic system is improved.

Description

Technical Field

[0001] This invention belongs to the field of intelligent traffic control technology, specifically relating to an intelligent traffic system control method and system based on type-two fuzzy logic and reinforcement learning. Background Art

[0002] Traditional traffic signal control methods, such as those based on fixed time intervals, struggle to adapt to complex and ever-changing traffic environments. Therefore, Intelligent Transportation Systems (ITS) have become crucial for solving these problems. While some existing ITS systems have incorporated fuzzy logic or reinforcement learning techniques, their performance in handling complex traffic conditions is poor due to the limitations of single fuzzy systems in handling uncertainty and the slow convergence and poor adaptability of single reinforcement learning algorithms. Summary of the Invention

[0003] The purpose of this invention is to provide a control method and system for intelligent transportation systems based on type-two fuzzy logic and reinforcement learning. By integrating type-two fuzzy logic models and reinforcement learning algorithms, it can effectively handle uncertainties in traffic flow and optimize traffic signal control strategies in real time, thereby improving the overall efficiency of the transportation system.

[0004] The present invention provides the following technical solutions:

[0005] Firstly, a control method for intelligent transportation systems based on type-two fuzzy logic and reinforcement learning is provided, including:

[0006] Collect real-time data from intelligent transportation systems;

[0007] The collected real-time data is processed using a type II fuzzy logic model, which outputs control signals for traffic lights and controls the traffic lights in real time.

[0008] The collected real-time data and the control signals output by the Type II fuzzy logic model are used as input variables, and the traffic flow prediction results are output using the established traffic flow prediction model.

[0009] Using real-time data from the traffic system, determine whether the current traffic light control strategy is effective. If so, use the DQN algorithm to optimize the traffic light control strategy; otherwise, use the FDQN algorithm to optimize the traffic light control strategy.

[0010] Based on the predicted traffic flow and the optimized traffic light control strategy, the control signals of the traffic lights are adjusted.

[0011] Optionally, the real-time data includes vehicle traffic flow, speed, lane occupancy rate, and vehicle queue length for the current phase and the next phase.

[0012] Optionally, the step of processing the collected real-time data using a type-two fuzzy logic model, outputting control signals for the traffic lights, and controlling the traffic lights in real time specifically includes:

[0013] The collected real-time data is subjected to single-point fuzzing.

[0014] For the fuzzy quantity after fuzzification, perform Mamdani fuzzy inference according to the set fuzzy inference rules, and output the fuzzy inference results after N rules are triggered;

[0015] The fuzzy reasoning result is defuzzified to output the control signal for the traffic light, wherein the control signal for the traffic light is the extension time of the green light.

[0016] Optionally, the fuzzy inference results after the N rules are triggered for:

[0017]

[0018] in, This is the fuzzy inference value after the l-th rule is triggered;

[0019] The fuzzy inference value after the lth rule is triggered for:

[0020]

[0021] in, The fuzzy inference value for the output type II consequent set. Let be the type II fuzzy membership value of the p-th input variable of the l-th rule;

[0022] The defuzzification representation of the fuzzy reasoning result is as follows:

[0023]

[0024] Among them, y cos (x) represents the defuzzified output value, M represents the total number of rules, and c l Let T be the centroid of the consequent set of the l-th rule, T represent the selected t-norm, and P be the total number of input variables. The fuzzy membership value is the type II fuzzy membership value of the i-th input of the l-th rule.

[0025] Optionally, the step of using real-time data from the traffic system to determine whether the current traffic light control strategy is effective, and if so, using the DQN algorithm to optimize the traffic light control strategy; otherwise, using the FDQN algorithm to optimize the traffic light control strategy, specifically includes:

[0026] The FDQN algorithm is based on the DQN algorithm. During training, the FDQN algorithm selects the output action a of the type-II fuzzy logic model as the action of the target network. f ;

[0027] The optimization objective formula for the FDQN algorithm is:

[0028] R t =W t -W t+1

[0029]

[0030] Among them, R t Let i be the reward value in the t-th period. t For the vehicles observed in period t, For vehicle i t The waiting time N in period t t W represents the total number of vehicles that cycle through in period t. t Let W be the total waiting time for all vehicles in period t. t+1 This represents the total waiting time for all vehicles during the t+1 period.

[0031] Optionally, the control method further includes: after optimizing the traffic signal control strategy using the FDQN algorithm, adjusting the type II fuzzy logic model and the FDQN algorithm online using feedback information from the traffic system.

[0032] Optionally, the online adjustment of the type-2 fuzzy logic model and the FDQN algorithm specifically includes:

[0033] The agent interacts with the environment to obtain a quadruple (s t ,a t ,R t ,s t+1 );

[0034] Based on quadruplets (s t ,a t ,R t ,s t+1 The network parameters of the current value network of the FDQN algorithm are updated by minimizing the loss function, so as to realize the online adjustment of the type II fuzzy logic model and the FDQN algorithm.

[0035] Specifically, when updating the network parameters of the current value network in the FDQN algorithm, the output action a of the type-II fuzzy logic model is used. f Obtain the estimated Q value of the policy update reward; based on the estimated Q value of the policy update reward, obtain the value function iteration process of the FDQN algorithm.

[0036] Optionally, the estimated Q-value Q of the obtained policy update reward is... T for:

[0037] Q T =R t +γQ target (s t+1 ,a f ,θ′)

[0038] Among them, R t Q represents the reward value for the current state, γ is the discount factor, and Q is the reward value for the current state. target (s t+1 ,a f ,θ′) is the target Q-value of the next-state target network, s t+1 For the next state, θ′ is the weight of the target network;

[0039] The iterative formula for the value function of the FDQN algorithm is:

[0040] Q estimation (s t ,a t ,θ)+α[Q T -Q estimation (s t ,a t ,θ)]→Q estimation (s t ,a t ,θ)

[0041] Among them, Q estimation (s t ,a t ,θ) is the current state s t The estimated Q value, θ is the network weight of the prediction network, a t It predicts the network's actions; α is the learning rate;

[0042] The loss function L(θ) of the FDQN algorithm is:

[0043] L(θ)=E[(R t +γQ target (s t+1 ,a f ,θ′)-Q estimation (s t ,a t ,θ)) 2 ]

[0044] Where E[] represents the expectation.

[0045] Optionally, the control method further includes: periodically training the type II fuzzy logic model, traffic flow prediction model, DQN algorithm, and FDQN algorithm offline.

[0046] Secondly, a control system for an intelligent transportation system based on type-two fuzzy logic and reinforcement learning is provided, comprising:

[0047] The data collection module is used to collect real-time data from the intelligent transportation system.

[0048] The real-time control module is used to process the collected real-time data using a type-two fuzzy logic model, output the control signal of the traffic lights, and control the traffic lights in real time.

[0049] The traffic flow prediction module takes the collected real-time data and the control signals output by the type II fuzzy logic model as input variables, and uses the established traffic flow prediction model to output traffic flow prediction results.

[0050] The control strategy optimization module is used to use real-time data from the traffic system to determine whether the current traffic light control strategy is effective. If it is, the DQN algorithm is used to optimize the traffic light control strategy; otherwise, the FDQN algorithm is used to optimize the traffic light control strategy.

[0051] The control signal adjustment module is used to adjust the control signals of traffic lights based on the predicted traffic flow and the optimized traffic light control strategy.

[0052] Compared with the prior art, the present invention has the following beneficial effects:

[0053] This invention uses a type-two fuzzy logic model to process the collected real-time data and outputs the control signal of the traffic lights. In addition, this invention optimizes the control signal of the traffic lights based on the prediction results of traffic flow and optimizes the control strategy of the traffic lights through a deep reinforcement learning algorithm. This invention integrates a type-two fuzzy logic system and a deep reinforcement learning algorithm, which can effectively handle the uncertainty in traffic flow and optimize the traffic signal control strategy in real time, thereby improving the overall efficiency of the traffic system. Attached Figure Description

[0054] Figure 1 This is a traffic decision-making principle diagram of the intelligent transportation system control method based on type-two fuzzy logic and reinforcement learning of the present invention;

[0055] Figure 2 This is a flowchart of the DQN algorithm of the present invention;

[0056] Figure 3 The figure shows the simulation results of the average cumulative reward values ​​of the FDQN algorithm and DQN algorithm of this invention.

[0057] Figure 4 This is a comparison chart of traffic flow information after four different control strategies were applied to control traffic signals under the same number of vehicles.

[0058] Figure 5 This is a flowchart of the FDQN algorithm of the present invention. Detailed Implementation

[0059] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0060] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0061] Example 1

[0062] like Figure 1 and Figure 5 As shown, a control method for intelligent transportation systems based on type-II fuzzy logic and reinforcement learning is provided, including the following steps:

[0063] S1: Collect real-time data from intelligent transportation systems.

[0064] Specifically, real-time data from intelligent transportation systems includes vehicle flow, speed, lane occupancy, and queue lengths for the current and next phases. Intelligent transportation systems can reference existing technologies.

[0065] Specifically, high-definition surveillance cameras were installed on both sides of the selected monitoring section. The surveillance data was acquired through the high-definition surveillance cameras, and then the surveillance data was processed by noise reduction, license plate recognition, color correction and other operations to export data such as queue length and vehicle speed, and processed into position matrix and speed matrix.

[0066] High-definition surveillance cameras cover different areas of the road to ensure the continuity and integrity of video surveillance; the installation positions and angles of these devices have been carefully planned and adjusted to achieve the best monitoring results.

[0067] S2: Uses a type II fuzzy logic model to process the collected real-time data, outputs the control signal for the traffic lights, and controls the traffic lights in real time.

[0068] Step S2 specifically includes the following steps:

[0069] S21: Perform single-point fuzzing on the collected real-time data.

[0070] Alternatively, the vehicle queue length of the current phase and the vehicle queue length of the next phase can be used as input for fuzzification.

[0071] S22: For the fuzzy quantity after fuzzification, perform Mamdani fuzzy inference according to the set fuzzy inference rules, and output the fuzzy inference results after N rules are triggered.

[0072] Specifically, the Type II fuzzy logic model includes a first-level fuzzy system and a second-level fuzzy system, which are used to handle uncertainties at different levels.

[0073] Specifically, the fuzzy quantity is obtained by performing Mamdani fuzzy inference on the predefined fuzzy inference rules, using the following formula:

[0074]

[0075] in, This is the fuzzy inference value after the l-th rule is triggered. The fuzzy inference value for the output type II consequent set. Let p be the type II fuzzy membership value of the p-th input variable. Let be the type II fuzzy membership value of the p-th input variable of the l-th rule.

[0076] If we perform single-point fuzzification on the input variables, the above equation simplifies to:

[0077]

[0078] Assuming that N out of M rules are triggered, the final reasoning result is... for:

[0079]

[0080] S23: Defuzzify the fuzzy reasoning result to output the control signal of the traffic light, wherein the control signal of the traffic light is the extension time of the green light.

[0081] Specifically, the defuzzified representation of the fuzzy inference result is as follows:

[0082]

[0083] Among them, y cos (x) represents the defuzzified output value, M represents the total number of rules, and c lLet T be the centroid of the consequent set of the l-th rule, T represent the selected t-norm, and P be the total number of input variables. The fuzzy membership value is the type II fuzzy membership value of the i-th input of the l-th rule.

[0084] Determining the membership degree values ​​of type II fuzzy systems requires determining the membership function. When determining the membership function, the influence of the membership function curve shape on the control performance of the control system must be considered. Since the Gaussian membership function curve has a relatively flat shape and control characteristics, and good stability, it is a reasonable form for describing fuzzy subsets. Therefore, the interval type II Gaussian membership function with uncertain deviation is selected, and the formula is:

[0085]

[0086] Where m is the center of the membership function, σ1 and σ2 are the two deviations of the membership function, and x is the input variable. The input variable is the type II fuzzy membership value.

[0087] The main structure of the Type II fuzzy controller is very similar to that of the Type I fuzzy controller. However, the output part of the Type I fuzzy system only has one defuzzification module, while the fuzzy set output by the Type II fuzzy inference engine is a Type II fuzzy output, which must first be downgraded to a Type I fuzzy output. Therefore, a downgrader needs to be added to the output part of the Type II fuzzy system before the defuzzification module.

[0088] As an example, Table 1 shows the inference rule table for a type-two fuzzy logic model. The input variables of the fuzzy control system are the vehicle queue length L1 in the current phase and the vehicle queue length L2 in the next phase at time t, and the output variable is the green light extension time T. The universe of discourse for the number of queued vehicles is set to [0, 50], the universe of discourse for the number of vehicle speeds is set to [0, 35], and the fuzzy universe of discourse for the output variable T is set to [0, 30]. The fuzzy set is divided into three fuzzy subsets based on the input values: short S, medium M, and long L.

[0089] Table 1. Inference Rules for Type II Fuzzy Logic Model

[0090]

[0091] S3: Using the collected real-time data and the control signals output by the Type II fuzzy logic model as input variables, and utilizing the established traffic flow prediction model, output the traffic flow prediction results.

[0092] Traffic flow prediction models are existing technical models for traffic flow prediction. As an example, the Multi-Graph Diffusion Attention Network (MGDAN) or the traffic flow prediction method based on blockchain and federated learning (TFPM-BFL) are used for traffic flow prediction.

[0093] S4: Using real-time data from the traffic system, determine whether the current traffic light control strategy is effective. If so, use the DQN algorithm to optimize the traffic light control strategy; otherwise, use the FDQN algorithm to optimize the traffic light control strategy.

[0094] Whether a traffic light control strategy is effective can be determined based on expert experience. Alternatively, it can be determined whether the queue length is less than a set threshold or whether the total waiting time is less than a set threshold.

[0095] Specifically, the control strategy of traffic lights is dynamically optimized using the FDQN (Fuzzy Deep Q-Network) algorithm, with the optimization objectives of maximizing traffic flow and reducing vehicle waiting time. The learning of the control strategy is guided by a designed reward function.

[0096] The FDQN algorithm is based on the DQN (Deep Q-Network) algorithm. During training, the FDQN algorithm selects the output action 'a' of the target network as the action of the type-II fuzzy logic model. f .

[0097] The optimization objective formula for the FDQN algorithm is:

[0098] R t =W t -W t+1

[0099]

[0100] Among them, R t Let i be the reward value in the t-th period. t For the vehicles observed in period t, For vehicle i t The waiting time N in period t t W represents the total number of vehicles that cycle through in period t. t Let W be the total waiting time for all vehicles in period t. t+1 This represents the total waiting time for all vehicles during the t+1 period.

[0101] like Figure 2 As shown, the FDQN algorithm is based on the DQN algorithm. When calculating the loss function, the DQN algorithm performs a max operation when selecting the target network output Q value. At this time, the target network output Q value is the maximum value, and Q can be obtained. target (s t+1 ,a t+1 ,θ′)=max a+1 Q target (s t+1 ,a t+1,θ′), while the target network output Q value selected by the FDQN algorithm is the output action a of the fuzzy control system. f (a f ∈{a on ,a off The output action a of the type II fuzzy logic model. f The action a of performing a max operation to obtain the Q value of the target network output. t+1 If there are unequal values, the corresponding target network output Q values ​​will also be unequal, then Q... target (s t+1 ,a f ,θ′)≤max a+1 Q target (s t+1 ,a t+1 Therefore, selecting the target network output Q value in the FDQN algorithm can reduce the error caused by the max operation in the DQN algorithm.

[0102] S5: Adjust the control signals of traffic lights based on the predicted traffic flow and the optimized traffic light control strategy.

[0103] To further ensure the accuracy of the control method, the Type II fuzzy logic model, traffic flow prediction model, DQN algorithm, and FDQN algorithm were all trained offline on a regular basis.

[0104] Example 2

[0105] A control method for intelligent transportation systems based on type-two fuzzy logic and reinforcement learning is provided. The difference between Embodiment 2 and Embodiment 1 is that, while retaining the features of Embodiment 1, the control method further includes the following steps:

[0106] S6: After optimizing the traffic signal control strategy using the FDQN algorithm, the type II fuzzy logic model and the FDQN algorithm are adjusted online using feedback information from the traffic system.

[0107] Online adjustments are made to the type-2 fuzzy logic system and the FDQN algorithm to improve the adaptability and robustness of the control system, such as enabling a rapid response when too many vehicles are detected in a queue or when traffic flow is heavy.

[0108] Step S6 specifically includes the following sub-steps:

[0109] S61: The agent interacts with the environment to obtain a quadruple (s t ,a t ,R t ,s t+1 ).

[0110] S62: Based on quadruples (s t ,at ,R t ,s t+1 The network parameters of the current value network of the FDQN algorithm are updated by minimizing the loss function, so as to realize the online adjustment of the type II fuzzy logic model and the FDQN algorithm model.

[0111] When updating the network parameters of the current value network in the FDQN algorithm, the output action a of the type II fuzzy logic model is used. f Obtain the estimated Q value of the policy update reward; based on the estimated Q value of the policy update reward, obtain the value function iteration process of the FDQN algorithm.

[0112] Specifically, the FDQN algorithm uses two independent state-action value functions. Instead of using a max operation during each update, it uses the fuzzy control system to output action 'a'. f Determine the target network action:

[0113]

[0114] The output action a using the type II fuzzy logic model f To obtain the estimated Q-value of the policy update return. T :

[0115]

[0116] Among them, R t Q represents the reward value for the current state, γ is the discount factor, and Q is the reward value for the current state. target (s t+1 ,a f ,θ′) is the target Q-value of the next-state target network, s t+1 For the next state, θ′ is the weight of the target network;

[0117] Based on the estimated Q-value of the policy update reward, the value function iteration process of the FDQN algorithm is obtained:

[0118] Q estimation (s t ,a t ,θ)+α[Q T -Q estimation (s t ,a t ,θ)]→Q estimation (s t ,a t ,θ)

[0119] Among them, Q estimation (s t ,a t ,θ) is the current state s t The estimated Q value, θ is the network weight of the prediction network, at It predicts the network's actions; α is the learning rate;

[0120] The loss function L(θ) of the FDQN algorithm is:

[0121] L(θ)=E[(R t +γQ target (s t+1 ,a f ,θ′)-Q estimation (s t ,a t ,θ)) 2 ]

[0122] Where E[] represents the expectation.

[0123] Example 3

[0124] Follow these steps to perform a real-world simulation of the road network environment simulated by SUMO using the TraCI interface, and conduct simulation analysis of different traffic demand conditions.

[0125] Step 1: Define the action space, state space, and reward space, and process the data using the FDQN algorithm. Input the action with the highest Q-value into the prediction network output, and use the quadruples obtained from the interaction between the action and the environment as the network training data.

[0126] Step 2: Perform fuzzy inference according to the set fuzzy rules, input the output action obtained by the fuzzy control system into the target network, and determine the output Q value of the target network.

[0127] Step 3: Calculate the loss function and update the prediction network weights using gradient descent. This method helps improve the online learning rate of the agent and increase the reward value for traffic light control actions.

[0128] Step 4: Continuously cycle through the process to obtain indicators such as vehicle waiting time, vehicle speed, and number of vehicle stops. Based on the results, adjust traffic light timings and regulate the timing of the next phase's traffic lights in real time to reduce road congestion as quickly as possible.

[0129] like Figure 3 As shown, the horizontal axis of the DQN and FDQN algorithms represents the training batch, and the vertical axis represents the average cumulative reward value. From Figure 3 As can be seen, in the initial stage of training, due to the lack of experience samples, the agent selects actions based on an exploration strategy in the environment, resulting in a small reward value for the algorithm. As training progresses, the agent continuously interacts with the environment and acquires a large number of experience samples, and the algorithm's reward value continuously increases, eventually reaching convergence at the 2000th simulation step.

[0130] like Figure 4 As shown, Figure 4(a) in the figure represents a comparison of vehicle queue lengths under the same traffic flow when four control strategies are used for traffic signal control. Figure 4 (b) in the figure represents a comparison of vehicle queuing speeds under the same traffic flow when four control strategies are used for traffic signal control. Figure 4 (c) in the figure represents a comparison of the average waiting time of vehicles under the same traffic flow when four control strategies are used for traffic signal control. Figure 4 (d) in the graph represents a comparison of the total waiting time for vehicles under the same traffic flow when four control strategies are used for traffic signal control; Figure 4 It is evident that this invention can significantly improve the intelligence level of traffic signal control systems, increase traffic flow, reduce congestion, and has important application value for urban traffic management.

[0131] Example 4

[0132] A control system for an intelligent transportation system based on type-two fuzzy logic and reinforcement learning is provided, comprising:

[0133] The data collection module is used to collect real-time data from the intelligent transportation system.

[0134] The real-time control module is used to process the collected real-time data using a type-two fuzzy logic model, output the control signal of the traffic lights, and control the traffic lights in real time.

[0135] The traffic flow prediction module takes the collected real-time data and the control signals output by the type II fuzzy logic model as input variables, and uses the established traffic flow prediction model to output traffic flow prediction results.

[0136] The control strategy optimization module is used to use real-time data from the traffic system to determine whether the current traffic light control strategy is effective. If it is, the DQN algorithm is used to optimize the traffic light control strategy; otherwise, the FDQN algorithm is used to optimize the traffic light control strategy.

[0137] The control signal adjustment module is used to adjust the control signals of traffic lights based on the predicted traffic flow and the optimized traffic light control strategy.

[0138] For a more detailed explanation of the above method, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.

[0139] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0140] Those skilled in the art will clearly understand that the techniques in the embodiments of the present invention can be implemented using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments of the present invention.

[0141] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A control method for an intelligent transportation system based on type-II fuzzy and reinforcement learning, characterized in that: include: Collect real-time data from intelligent transportation systems; A type-II fuzzy logic model is used to process the collected real-time data, output the control signal of the traffic light, and control the traffic light in real time; The collected real-time data and the control signal output by the type-II fuzzy logic model are used as input variables, and the established traffic flow prediction model is used to output the traffic flow prediction results; Use the real-time data of the traffic system to determine whether the current traffic light control strategy is effective. If so, use the DQN algorithm to optimize the traffic light control strategy. If not, use the FDQN algorithm to optimize the traffic light control strategy. Based on the predicted traffic flow prediction results and the optimized traffic light control strategy, the control signal of the traffic light is adjusted.

2. The intelligent transportation system control method based on type-two fuzzy and reinforcement learning according to claim 1 is characterized in that: The real-time data includes vehicle flow, speed, lane occupancy, and vehicle queue lengths in the current phase and the next phase.

3. The intelligent transportation system control method based on type-two fuzzy and reinforcement learning according to claim 1 is characterized in that: The method of using a type-II fuzzy logic model to process the collected real-time data, output a control signal for a traffic light, and control the traffic light in real time specifically includes: Perform single-point fuzzification operations on the collected real-time data; For the fuzzy quantity after fuzzification operation, Mamdani fuzzy reasoning is performed according to the set fuzzy reasoning rules, and the fuzzy reasoning results after N rules are triggered are output; The fuzzy inference result is defuzzified to output a control signal of the traffic light, where the control signal of the traffic light is the extension time of the green light.

4. The intelligent transportation system control method based on type-two fuzzy and reinforcement learning according to claim 3 is characterized in that: The fuzzy reasoning results after the N rules are triggered for: in, is the fuzzy inference value after the lth rule is triggered; The fuzzy inference value after the lth rule is triggered for: in, is the fuzzy inference value of the output type-two consequent set, is the type-II fuzzy membership value of the p-th input variable of the l-th rule; The defuzzification of the fuzzy reasoning result is expressed as: Among them, y cos (x) represents the output value after defuzzification, M represents the total number of rules, c l is the center of gravity of the consequent set of the lth rule, T represents the selected t-norm, P is the total number of input variables, is the type-II fuzzy membership value of the ith input of the lth rule.

5. The intelligent transportation system control method based on type-two fuzzy and reinforcement learning according to claim 1 is characterized in that: The method uses the real-time data of the traffic system to determine whether the current control strategy of the traffic light is effective. If so, the DQN algorithm is used to optimize the control strategy of the traffic light. If not, the FDQN algorithm is used to optimize the control strategy of the traffic light. Specifically, the method includes: The FDQN algorithm is based on the DQN algorithm. The action of the target network selected by the FDQN algorithm during training is the output action a of the type-two fuzzy logic model. f ; The optimization objective formula of the FDQN algorithm is: R t =W t -W t+1 Among them, R t is the reward value in the tth cycle, i t is the vehicle observed in the tth period, For vehicle i t Waiting time in cycle t, N t is the total number of vehicles circulating in the tth cycle, W t is the total waiting time of all vehicles in the tth cycle, W t+1 is the total waiting time of all vehicles in the t+1th cycle.

6. The intelligent transportation system control method based on type-two fuzzy and reinforcement learning according to claim 1 is characterized in that: The control method further comprises: after optimizing the traffic light control strategy by using the FDQN algorithm, using feedback information from the traffic system to perform online adjustment on the type II fuzzy logic model and the FDQN algorithm.

7. The intelligent transportation system control method based on type-two fuzzy and reinforcement learning according to claim 6 is characterized in that: The online adjustment of the type-2 fuzzy logic model and the FDQN algorithm specifically includes: The agent interacts with the environment and obtains a quaternion (s t ,a t ,R t ,s t+1 ); Based on the four-tuple (s t , a t , R t ,s t+1 ), by minimizing the loss function, the network parameters of the current value network of the FDQN algorithm are updated to achieve online adjustment of the type-2 fuzzy logic model and the FDQN algorithm; Specifically, when updating the network parameters of the current value network in the FDQN algorithm, the output action a of the type-2 fuzzy logic model is used. f , obtain the estimated Q value of the strategy update return; according to the estimated Q value of the strategy update return, obtain the value function iteration process of the FDQN algorithm.

8. The intelligent transportation system control method based on type-two fuzzy and reinforcement learning according to claim 7 is characterized in that: The acquisition strategy updates the estimated Q value Q of the reward T for: Q T =R t +γQ target (s t+1 ,a f ,θ′) Among them, R t is the reward value of the current state, γ is the discount factor, Q target (s t+1 , a f ,θ′) is the target Q value of the target network in the next state, s t+1 is the next state, θ′ is the weight of the target network; The value function iteration formula of the FDQN algorithm is: Q estimation (s t ,a t ,θ)+α[Q T -Q estimation (s t ,a t ,θ)]→Q estimation (s t ,a t ,i) Among them, Q estimation (s t , a t ,θ) is the current state s t The estimated Q value of θ is the network weight of the prediction network, a t is the action of the prediction network; α is the learning rate; The loss function L(θ) of the FDQN algorithm is: L(θ)=E[(R t +γQ target (s t+1 ,a f ,θ′)-Q estimation (s t ,a t ,θ)) 2 ] Among them, E[] is the expectation.

9. The intelligent transportation system control method based on type-two fuzzy and reinforcement learning according to claim 1 is characterized in that: The control method further comprises: regularly training the type II fuzzy logic model, the traffic flow prediction model, the DQN algorithm and the FDQN algorithm offline.

10. A control system for an intelligent transportation system based on type-II fuzzy and reinforcement learning, characterized in that: include: A collection module, used to collect real-time data of intelligent transportation systems; A real-time control module, for processing the collected real-time data using a type-II fuzzy logic model, outputting a control signal for a traffic light, and controlling the traffic light in real time; The traffic flow prediction module is used to use the collected real-time data and the control signal output by the type-II fuzzy logic model as input variables, and output the traffic flow prediction results using the established traffic flow prediction model; The control strategy optimization module is used to use the real-time data of the traffic system to determine whether the current control strategy of the traffic light is effective. If so, the DQN algorithm is used to optimize the control strategy of the traffic light. If not, the FDQN algorithm is used to optimize the control strategy of the traffic light. The control signal adjustment module is used to adjust the control signal of the traffic light based on the predicted traffic flow prediction result and the optimized control strategy of the traffic light.

Images