Traffic congestion time prediction method, device, electronic device and storage medium

Abstract

The present invention discloses a traffic congestion time prediction method, device, electronic device and storage medium. The traffic congestion time prediction method includes: obtaining current environmental status information; determining the current instantaneous feedback information corresponding to the current environmental status information; training a traffic congestion time prediction model to be trained based on the current environmental status information and the current instantaneous feedback information to obtain a trained traffic congestion time prediction model, and the trained traffic congestion time prediction model is used to predict traffic congestion time. The above technical solution determines the current instantaneous feedback information corresponding to the current environmental status information through the current environmental status information, and trains the traffic congestion time prediction model to be trained in combination with the current environmental status information and the current instantaneous feedback information, thereby improving the credibility of the training of the traffic congestion time prediction model, and thereby improving the sensitivity of the traffic congestion time prediction process to current environmental changes.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of intelligent transportation technology, and in particular to a method, device, electronic device, and storage medium for predicting traffic congestion time. Background Art

[0002] With the continuous development of society and the increasing demand for travel, while the number and quality of motor vehicles continue to break through and innovate, traffic congestion has also arisen. Traffic congestion brings many inconveniences to people's daily lives and work, and greatly increases urban traffic pressure. Therefore, traffic congestion time prediction models can be used to predict traffic congestion time and thus alleviate urban traffic pressure.

[0003] In existing technologies, the training of traffic congestion time prediction models is primarily based on historical data. By fitting the models to past historical experience, future traffic congestion times are predicted. However, due to the lack of reference to historical data and experience, existing models cannot effectively adapt to rapidly changing traffic congestion conditions for occasional traffic congestion, resulting in large errors in the predicted traffic congestion times. Therefore, improving the reliability of traffic congestion time prediction model training and making the prediction process more sensitive to current environmental changes are urgent issues that need to be addressed. Summary of the Invention

[0004] The present invention provides a traffic congestion time prediction method, device, electronic device and storage medium, which can effectively deal with occasional traffic congestion, improve the credibility of traffic congestion time prediction model training, and further improve the sensitivity of the traffic congestion time prediction process to current environmental changes.

[0005] In a first aspect, an embodiment of the present invention provides a method for predicting traffic congestion time, comprising:

[0006] Get current environment status information;

[0007] Determining current instant feedback information corresponding to the current environmental status information;

[0008] The traffic congestion time prediction model to be trained is trained based on the current environment state information and the current immediate feedback information to obtain a trained traffic congestion time prediction model, and the trained traffic congestion time prediction model is used to predict traffic congestion time.

[0009] In a second aspect, an embodiment of the present invention provides a driving parameter prediction method, comprising:

[0010] Inputting environmental state information into a trained traffic congestion time prediction model to obtain action information and predicted traffic congestion time, wherein the traffic congestion time prediction model is trained based on the method described in the first aspect;

[0011] determining a processing strategy based on the action information;

[0012] The driving parameters include the action information, the traffic congestion time and the processing strategy.

[0013] In a third aspect, an embodiment of the present invention provides a traffic congestion time prediction device, comprising:

[0014] Acquisition module, used to obtain current environment status information;

[0015] A first determining module is used to determine the current instant report information corresponding to the current environmental status information;

[0016] A training module is used to train the traffic congestion time prediction model to be trained based on the current environmental state information and the current immediate feedback information to obtain a trained traffic congestion time prediction model, and the trained traffic congestion time prediction model is used to predict traffic congestion time.

[0017] In a fourth aspect, an embodiment of the present invention provides a driving parameter prediction device, comprising:

[0018] An input module, configured to input environmental state information into a trained traffic congestion time prediction model to obtain action information and predicted traffic congestion time, wherein the traffic congestion time prediction model is trained based on the method described in the first aspect;

[0019] a second determining module, configured to determine a processing strategy based on the action information;

[0020] The driving parameters include the action information, the traffic congestion time and the processing strategy.

[0021] In a fifth aspect, an embodiment of the present invention provides an electronic device, including:

[0022] at least one processor; and

[0023] a memory communicatively connected to the at least one processor; wherein,

[0024] The memory stores a computer program executable by the at least one processor. The computer program is executed by the at least one processor to enable the at least one processor to perform the method according to the first aspect or the method according to the second aspect.

[0025] In a sixth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect, or implements the method described in the second aspect.

[0026] The technical solution of the embodiment of the present invention determines the current instantaneous report information corresponding to the current environmental state information through the current environmental state information, and trains the traffic congestion time prediction model to be trained in combination with the current environmental state information and the current instantaneous report information, thereby improving the credibility of the traffic congestion time prediction model training, and further improving the sensitivity of the traffic congestion time prediction process to current environmental changes.

[0027] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0029] Figure 1 It is a schematic diagram of the structure of the Critic network implementing the present invention;

[0030] Figure 2 This is a flow chart of a method for predicting traffic congestion time according to the first embodiment of the present invention;

[0031] Figure 3 This is a flow chart of a method for predicting traffic congestion time according to a second embodiment of the present invention;

[0032] Figure 4 This is a flow chart of a driving parameter prediction method provided according to the third embodiment of the present invention;

[0033] Figure 5 This is a schematic diagram of the structure of a traffic congestion time prediction device provided according to a fourth embodiment of the present invention;

[0034] Figure 6 This is a schematic structural diagram of a driving parameter prediction device provided according to a fifth embodiment of the present invention;

[0035] Figure 7 A schematic structural diagram of an electronic device that can be used to implement an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0036] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0037] It should be noted that the terms "first," "second," and the like in the present invention are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to these processes, methods, products, or apparatuses.

[0038] It is understandable that before using the technical solutions disclosed in the embodiments of the present invention, the type, scope of use, and usage scenarios of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0039] Traffic congestion duration estimation has long been a hot topic and a challenging subject for research in both academia and industry. Traffic congestion is a dynamic process, with congestion duration varying with changing traffic conditions. Accurately estimating traffic congestion duration in this dynamic environment remains a pressing challenge. In recent years, with the rapid advancements in high-performance computing, big data, and deep learning technologies, reinforcement learning has also garnered increased attention and rapid development. In particular, deep reinforcement learning (DRL), a combination of reinforcement learning and deep learning, has achieved breakthroughs and demonstrated powerful learning and decision-making capabilities in various fields. Consequently, numerous researchers have incorporated DRL-related methods into the study of traffic state changes and control, achieving significant progress. However, the current challenge is that traffic congestion duration estimation based on DRL-related models is a static process. Dynamic estimation of traffic congestion duration requires models that can respond promptly to changing traffic conditions.

[0040] The present invention provides a traffic congestion time prediction method, device, electronic device and storage medium, which are based on deep reinforcement learning as a basic framework. In order to solve the problem that existing deep reinforcement learning algorithms cannot accurately estimate dynamically changing congestion time, the current instant reward information is used to improve the existing deep reinforcement learning algorithm when training the traffic congestion time prediction model, thereby improving the credibility of the traffic congestion time prediction model training and further improving the sensitivity of the traffic congestion time prediction process to current environmental changes.

[0041] By using the traffic congestion time prediction method provided by the present invention to train a traffic congestion time prediction model, and applying the trained traffic congestion time prediction model to the traffic congestion time prediction process, the traffic congestion time can be predicted under daily traffic conditions (such as regular congestion), and the traffic congestion time can also be predicted when abnormal traffic conditions occur (such as occasional congestion).

[0042] In the present invention, before training the traffic congestion time prediction model, a traffic congestion time prediction model to be trained is first constructed. When constructing the traffic congestion time prediction model to be trained, it includes but is not limited to constructing a learning framework and constructing the traffic congestion time prediction model to be trained.

[0043] First, a learning framework is constructed. The present invention does not limit the method of constructing the learning framework, as long as the learning framework can be constructed. The learning framework in the present invention can be based on the Markov reinforcement learning framework, which regards the traffic system as the environment and the vehicles as intelligent agents. The agents can obtain the current environmental state information in real time and predict the traffic congestion time based on the current environmental state information. The model that describes the traffic congestion time prediction process using the Markov model specifically includes:

[0044] (1) State space S: It can include environmental state information such as time, road section, number of lanes, current vehicle speed, length of congestion time, speed of other vehicles on the same road section, historical time of passing the road section, length of the road section and distance to the congestion point, etc., which together constitute the environmental state variables of the model.

[0045] (2) Action space A: The action space in the present invention can be the vehicle speed, which can be a continuous variable greater than or equal to 0. After executing an action, it will transfer to a new state. For example, after executing an action, it will correspond to new environmental state information.

[0046] (3) Reward function R: The reward function of the traffic congestion time prediction model can be designed as the accuracy of the congestion time prediction. The accuracy of the congestion time prediction can be evaluated by the evaluation index R 2 Indicates that the evaluation index R 2 The larger the value, the higher the accuracy of the congestion time prediction.t The calculation formula is:

[0047]

[0048] in, It can refer to the traffic congestion time predicted by the traffic congestion time prediction model at time i, where i can be any time from time i=1 to time i=n; y i It can refer to the actual traffic congestion time at time i; It may refer to the average value of the actual traffic congestion time at all times from time i=1 to time i=n.

[0049] (4) Discounted return rate γ: the value space is [0,1].

[0050] Secondly, a traffic congestion time prediction model to be trained is constructed. The traffic congestion time prediction model to be trained constructed by the present invention can be based on the Deep Deterministic Policy Gradient algorithm (DDPG). The output of this algorithm is a precise deterministic action, which can reduce randomness and reduce errors. The DDPG algorithm can be composed of a policy network (i.e., Actor network) and a value network (i.e., Critic network), wherein the Actor network (i.e., the first network in the present invention) outputs a deterministic action (i.e., vehicle speed), and the Critic network (i.e., the second network in the present invention) can be used to evaluate the Actor output action (i.e., traffic congestion time). Each network can be further divided into a reality network and a target network (i.e., Target network).

[0051] The Actor and Critic in the present invention may refer to two deep neural networks, respectively represented by μ(s|θ μ ) and Q(s,a|θ Q ) represents, where a=μ(s|θ μ ), a represents action, s represents state, that is, Q(s,a|θ Q ) can be expressed as Q(s,μ(s|θ μ )|θ Q ); build the Target network of Actor and Critic at the same time, using μ'(s|θ μ ') and Q'(s,a'|θ Q ') represents, where a'=μ'(s|θ μ '), that is, Q'(s,a'|θ Q ') can be expressed as Q'(s,μ'(s|θ μ ')|θ Q ').θ μ and θ μ' is the parameter of the Actor network, θ Q and θ Q ' is the parameter of the Critic network. μ(s|θ μ ) and μ'(s|θ μ ') The structure of the network can be exactly the same, but the parameter values ​​are different. Q ) and Q'(s,a'|θ Q ') The structure of the network can be exactly the same, but the parameter values ​​are different.

[0052] The two deep neural networks, Actor and Critic, in the present invention can each include one input layer (InputLayer), three LSTM layers, one batch normalization layer (BatchNormalization) and two fully connected layers (Dense). The input dimension of the input layer of the Actor network can be 30*E (where 30 can be the environmental state information of the past 30 cycles, such as: time, road section, number of lanes, current speed, length of congestion time, speed of other vehicles on the same road section, historical time passing through the road section, length of the road section and distance to the congestion point, etc., and E can represent the dimension of the state space); each LSTM layer can extract the features of the input sequence data and perform nonlinear processing on the extracted features through the rectified linear unit (RELU) activation function; the output of the last LSTM layer is normalized by the BatchNormalization layer before inputting into the fully connected layer, where the meaning of normalization is to convert the data into data with a mean of 0 and a variance of 1; the first fully connected layer also uses the RELU activation function to perform another feature transformation on the data input to the first fully connected layer so that it can be connected to the output layer; the second fully connected layer is the output layer, and the output can be a continuous value greater than or equal to 0, and no activation function is required.

[0053] Figure 1 It is a schematic diagram of the structure of the Critic network of the present invention. Figure 1As shown, the input dimension of the Critic network's input layer is 30*(E+1), where 30 represents the environment state information for the past 30 cycles. (E+1) can contain E dimensions of the state space and the output of one Actor network. The output of the Actor network is a continuous value greater than or equal to 0. After being expanded to a 30*1 tensor, it is concatenated with the state tensor to obtain a 30*(E+1) tensor, which is then input into the Critic network. The three LSTM layers, one BatchNormalization layer, and two fully connected layers of the Critic network perform the same functions as those in the Actor network. The second fully connected layer is the output layer, which outputs the next estimated value of the next environment state information.

[0054] It should be noted that the deep neural network in the present invention is not limited to the deep neural network with the above structure. Other deep neural network structures that can achieve the same function are also acceptable, and this is not limited in the present invention.

[0055] It should be noted that the traffic congestion time prediction model to be trained constructed by the present invention can also be based on other reinforcement learning algorithms, such as action evaluation (Actor-Critic, AC), advantage action evaluation (Advantage Actor-Critic, A2C), trust region policy optimization (Trust Region Policy Optimization, TRPO), proximal policy optimization (Proximal Policy Optimization, PPO) and other single-agent reinforcement learning algorithms. The improved algorithm is also suitable for guiding the traffic congestion time prediction process.

[0056] Example 1

[0057] Figure 2 This is a flow chart of a traffic congestion time prediction method provided according to the first embodiment of the present invention. This embodiment is applicable to the case of training a traffic congestion time prediction model. The method can be executed by a traffic congestion time prediction device. The traffic congestion time prediction device can be implemented in the form of software and / or hardware and integrated into an electronic device. Furthermore, the electronic device includes but is not limited to: computers, laptops, smart phones, servers, etc. Figure 1 As shown, the method includes:

[0058] S110: Obtain current environment status information.

[0059] Current environmental state information may refer to the environmental state information currently used to train the traffic congestion time prediction model. Environmental state information may refer to information representing the state of an environment. Current environmental state information may reflect the state of the vehicle's current environment, where the state may be a description of the vehicle's current environment. There are no limitations on the current environmental state information, as long as it reflects the state of the vehicle's current environment. Specifically, the current environmental state information may be determined based on the actual vehicle's environment.

[0060] In one embodiment, the vehicle is currently in a congested section, and the corresponding current environmental status information may include but is not limited to: time, section, number of lanes, current vehicle speed, length of congestion time, speed of other vehicles on the same section, historical time passing through the section, length of the section, and distance to the congestion point, etc.

[0061] There is no limitation on the method of obtaining the current environmental status information, as long as the current environmental status information can be obtained. For example, the changes in the current environment of the vehicle can be monitored by devices such as on-board cameras or sensors, and the data monitored by devices such as on-board cameras or sensors can be uploaded to electronic devices, thereby obtaining the current environmental status information, wherein the on-board cameras or sensors can be cameras or sensors selected according to actual needs; for example, the current environmental status information can be obtained by an electronic device through a human-computer interaction device of the electronic device, wherein the human-computer interaction device can be a device for the interactor (i.e., the driver) to perform human-computer interaction. The human-computer interaction device includes but is not limited to a touch screen and input components (such as keyboards and buttons, etc.). An interface for inputting the current environmental status information can be displayed on the human-computer interaction device, and the current environmental status information can be input on the interface for inputting the current environmental status information, such as the time, the road section, the number of lanes, or the current vehicle speed.

[0062] S120: Determine current instant report information corresponding to the current environmental status information.

[0063] The current instant reward information may refer to information representing the instant reward at the current moment. The current instant reward information may include but is not limited to the reward at the current moment. t , where the current reward r t It can be the cumulative discount of the reward calculated by the reward function. The reward function can be a function set according to actual needs. For example, the reward function can be set as the accuracy of the congestion time prediction. The accuracy of the congestion time prediction can be evaluated by the evaluation index R 2 Indicates that the evaluation index R 2 The larger the value, the higher the accuracy of the congestion time prediction. t The calculation formula is:

[0064]

[0065] The method for determining the current instantaneous reward information corresponding to the current environmental state information is not limited, as long as the current instantaneous reward information corresponding to the current environmental state information can be determined. For example, the current instantaneous reward information corresponding to the current environmental state information can be determined by inputting the current environmental state information into the traffic congestion time prediction model to be trained. That is, under the current environmental state information, the current action information can be calculated according to the first network in the traffic congestion time prediction model to be trained, and the current action information can be executed to obtain the reward r at the current moment. t , and then the current immediate feedback information corresponding to the current environmental state information can be determined. Among them, the current action information can refer to information representing the action at the current moment, such as the current vehicle speed.

[0066] S130 , training the to-be-trained traffic congestion time prediction model based on the current environment state information and the current immediate feedback information to obtain a trained traffic congestion time prediction model, and using the trained traffic congestion time prediction model to predict traffic congestion time.

[0067] The specific method for training the traffic congestion time prediction model to be trained based on the current environmental state information and the current immediate feedback information to obtain the trained traffic congestion time prediction model is not limited, as long as the trained traffic congestion time prediction model can be obtained. For example, in each round of training, the traffic congestion time prediction model can be trained based on each moment within a set time period. Each moment within the set time period can include the current moment corresponding to the current environmental state information and historical moments before the current moment.

[0068] At each moment, the instantaneous return information and correction coefficient corresponding to that moment can be calculated, and based on the instantaneous return information and correction coefficient corresponding to that moment, combined with the next estimated value of the next environmental state information, the state estimate at each moment is determined; the state estimate at each moment is weighted averaged to determine the target estimated value of the next environmental state information; the network parameters are updated based on the target estimated value so that the updated network parameters can be used in the next round of training; the above steps can be a training process for one training round. After completing one training round, the operation of obtaining the current environmental state information can be returned to continue multiple rounds of training until the end condition is met, and the trained traffic congestion time prediction model can be obtained. The trained traffic congestion time prediction model can be used to predict traffic congestion time. For example, the output of the trained traffic congestion time prediction model can be used to predict traffic congestion time.

[0069] The next environmental state information may refer to information representing the state of the environment at the next moment, and the next estimated value may refer to an estimated value determined based on the next environmental state information. The next estimated value may be an estimated value for the next moment calculated by the second network; the target estimated value may refer to an estimated value determined based on multiple state estimates at the current moment corresponding to the current environmental state information and at multiple historical moments prior to the current moment. The termination condition may refer to a condition that terminates the model training process.

[0070] The technical solution of the embodiment of the present invention determines the current instantaneous report information corresponding to the current environmental state information through the current environmental state information, and trains the traffic congestion time prediction model to be trained in combination with the current environmental state information and the current instantaneous report information. The current instantaneous report information is taken into account in the process of training the traffic congestion time prediction model to be trained, making the training process more reasonable, improving the credibility of the traffic congestion time prediction model training, and thereby improving the sensitivity of the traffic congestion time prediction process to current environmental changes.

[0071] Example 2

[0072] Figure 3 This is a flowchart of a traffic congestion time prediction method provided according to Example 2 of the present invention. This embodiment is based on the above-mentioned Example 1, and trains the current immediate feedback information corresponding to the current environmental state information and the traffic congestion time prediction model to be trained based on the current environmental state information and the current immediate feedback information, thereby further refining the trained traffic congestion time prediction model.

[0073] In an embodiment of the present invention, determining the current instantaneous feedback information corresponding to the current environmental state information includes: inputting the current environmental state information into a first network included in a traffic congestion time prediction model to be trained to determine current action information; and executing the current action information to obtain the current instantaneous feedback information corresponding to the current environmental state information.

[0074] In an embodiment of the present invention, a traffic congestion time prediction model to be trained is trained based on current environmental state information and current instantaneous report information to obtain a trained traffic congestion time prediction model, including: determining a current correction coefficient based on the current environmental state information and the current instantaneous report information; determining a state estimation value for each moment within a set time period based on the current correction coefficient, the current instantaneous report information, historical information, a required first target parameter and a next estimation value, wherein the moments within the set time period include the current moment corresponding to the current environmental state information and historical moments before the current moment, and the historical information includes instantaneous report information corresponding to the historical moments and correction coefficients corresponding to the historical moments; determining a target estimation value for the next environmental state information based on a weighted average of the state estimation values; determining network parameters based on the target estimation value and the required second target parameter, and returning to the operation of obtaining the current environmental state information to continue training the traffic congestion time prediction model to be trained until the end condition is met to obtain the trained traffic congestion time prediction model.

[0075] like Figure 3 As shown, the method includes:

[0076] S110: Obtain current environment status information.

[0077] S121. Input current environment state information into a first network included in a traffic congestion time prediction model to be trained to determine current action information.

[0078] The current action information may refer to information representing the action at the current moment, such as the vehicle speed at the current moment. The first network included in the traffic congestion time prediction model to be trained may be the Actor network mentioned in the present invention when constructing the traffic congestion time prediction model to be trained. The output of the Actor network is a deterministic action. By inputting the current environmental state information into the first network included in the traffic congestion time prediction model to be trained, the current action information can be determined. The calculation formula is:

[0079] a t =μ(s t |θ μ )+Ω t

[0080] Among them, a t is the current action information; θ μ is the parameter of the first network; s t Indicates the current environment status information; Ω t It can be an error matrix with mean 0 and variance 1.

[0081] S122: Execute the current action information to obtain the current immediate feedback information corresponding to the current environment state information.

[0082] The method for executing the current action information is not limited, as long as the current action information can be executed. For example, the current action information can be executed by an electronic device through programming software in the electronic device. The programming software can be selected according to actual needs, as long as the software can execute the current action information.

[0083] Execute the current action information and get the current immediate feedback information corresponding to the current environment state information. It can be understood that the current environment state information s t Under the current environment, the electronic device interacts with the current state information to obtain the current instant feedback information r t .

[0084] S131. Determine a current correction coefficient based on current environmental status information and current instant feedback information.

[0085] Among them, the current correction coefficient can refer to the correction coefficient corresponding to the current moment. The current correction coefficient can be used to correct the next estimated value at the next moment, so that the next estimated value at the next moment is more accurate and has better timeliness, and can make a more timely response in the event of occasional congestion.

[0086] There is no limitation on the method of determining the current correction coefficient based on the current environmental state information and the current instant feedback information, as long as the current correction coefficient can be determined based on the current environmental state information and the current instant feedback information. For example, the current action information may be executed under the current environmental state information to obtain the next environmental state information, and then the current estimated value of the current environmental state information and the next estimated value of the next environmental state information are calculated using the second network included in the traffic congestion time prediction model to be trained, and the current correction coefficient is determined by combining the current estimated value, the next estimated value and the current instant feedback information. Among them, the second network included in the traffic congestion time prediction model to be trained may refer to the Critic network mentioned in the present invention. The current estimated value may refer to the estimated value at the current moment calculated by the second network. The next estimated value may be for the current estimated value, and the next estimated value may be the estimated value at the next moment corresponding to the current moment.

[0087] Furthermore, based on the current environmental status information and the current instant feedback information, the current correction coefficient is determined, including:

[0088] executing current action information determined by a first network included in the traffic congestion time prediction model to be trained to obtain next environment state information;

[0089] Inputting the current environment state information and the current action information into a second network included in the traffic congestion time prediction model to be trained to obtain a current estimated value of the current environment state information;

[0090] Inputting the next environmental state information and the next action information corresponding to the next environmental state information into a second network included in the traffic congestion time prediction model to be trained to obtain a next estimated value of the next environmental state information;

[0091] A current correction factor in the correction factor is determined based on the current estimate, the next estimate, and the current instantaneous return information.

[0092] The current action information determined by the first network included in the traffic congestion time prediction model to be trained is executed to obtain the next environmental state information, which can be understood as: t Under the current state, the electronic device interacts with the current environment state information and executes the current action information a t , get the next environment state information s t+1 .

[0093] The current environment state information and the current action information are input into the second network included in the traffic congestion time prediction model to be trained to obtain the current estimated value of the current environment state information. It can be understood that the current environment state information s t and current action information a t As the input parameter of the second network, the current estimated value Q'(s t ,μ'(s t |θ μ ')|θ Q ').

[0094] The next environmental state information and the next action information corresponding to the next environmental state information are input into the second network included in the traffic congestion time prediction model to be trained to obtain the next estimated value of the next environmental state information. It can be understood that the next environmental state information s t+1 and next action information a t+1 As the input parameter of the second network, the next estimated value Q'(s t+1 ,μ'(s t+1 |θ μ ')|θ Q ').

[0095] The method for determining the current correction factor in the correction factor based on the current estimated value, the next estimated value, and the current instantaneous return information is not limited, as long as the current correction factor in the correction factor can be determined based on the current estimated value, the next estimated value, and the current instantaneous return information. For example, the current correction factor in the correction factor can be determined through a mathematical operation based on the current estimated value, the next estimated value, and the current instantaneous return information.

[0096] Based on the current estimated value, the next estimated value and the current instant return information, the current correction coefficient in the correction coefficient is determined. It can be understood that different correction coefficients correspond to different moments. The current correction coefficient β corresponding to the current moment can be determined by the current estimated value, the next estimated value and the current instant return information. t .

[0097] Determining the current correction coefficient based on the current environmental status information and the current instant feedback information can facilitate the subsequent correction of the next estimated value at the next moment by the current correction coefficient, making the next estimated value at the next moment more accurate.

[0098] In one embodiment, Q(s) can be calculated by the real network in the second network. t ,a|θ Q ) and Q(s t+1 ,a|θ Q ), where Q(s t ,a|θ Q ) can represent the current state value corresponding to the current moment, that is, the value of the state at the current moment, Q(s t+1 ,a|θ Q ) can represent the next state value corresponding to the next moment of the current moment, that is, the value of the state at the next moment; Q'(s t ,μ'(s t |θ μ ')|θ Q ') and Q'(s t+1 ,μ'(s t+1 |θ μ ')|θ Q ').

[0099] Furthermore, based on the current estimated value, the next estimated value, and the current instant return information, a current correction factor in the correction factor is determined, including:

[0100] Determine the difference between the current estimate and the next estimate;

[0101] The ratio of the current instant return information to the difference is determined as the current correction coefficient in the correction coefficient.

[0102] Determine the difference between the current estimate and the next estimate, that is, the difference between the current estimate and the next estimate is Q'(s t ,μ'(s t |θ μ ')|θ Q ')-Q'(s t+1 ,μ'(s t+1 |θ μ ')|θ Q ');

[0103] The current instant feedback information r t The ratio of the difference is determined as the current correction coefficient in the correction coefficient, that is, the current correction coefficient β t The calculation formula is:

[0104] β t =r t / (Q'(s t ,μ'(s t |θ μ ')|θ Q ')-Q'(s t+1 ,μ'(s t+1 |θ μ ')|θ Q '))

[0105] In one embodiment, (s t ,a t ,r t ,s t+1 ,β t ) is saved to the experience buffer Buffer, wherein the experience buffer Buffer can be used to store the (s t ,a t ,r t ,s t+1 ,β t ) data, and then in subsequent training, the data stored in the experience buffer Buffer can be called to train the traffic congestion time prediction model. The length of the experience buffer Buffer is not limited and can be a length set according to actual needs. When the length of the experience buffer Buffer exceeds a set length threshold K, the data first stored in the experience buffer Buffer is removed to facilitate the storage of further data in the experience buffer Buffer, wherein the set length threshold K can be a length set according to actual needs.

[0106] S132. Determine the state estimation value at each moment within the set time period based on the current correction coefficient, the current instantaneous feedback information, the historical information, the required first target parameter and the next estimation value.

[0107] The moments within the set time period include the current moment corresponding to the current environmental status information and the historical moments before the current moment. The historical information includes the immediate feedback information corresponding to the historical moments and the correction coefficients corresponding to the historical moments.

[0108] There is no limitation on the set duration, and the set duration can be set according to actual needs, as long as the moments within the set duration can include the current moment corresponding to the current environmental status information and the historical moments before the current moment. The historical moment is relative to the current moment corresponding to the current environmental status information, and the historical moment can be a general term for the moments before the current moment. The historical information includes the instantaneous return information corresponding to the historical moment and the correction coefficient corresponding to the historical moment. It can be understood that within the historical moment, each moment corresponding to the historical moment has corresponding instantaneous return information and correction coefficient. The historical information is the complete information of the instantaneous return information and correction coefficient corresponding to each moment within the historical moment.

[0109] Furthermore, based on the current correction coefficient, the current instantaneous return information, the historical information, the required first target parameter and the next estimated value, the state estimated value at each moment within the set time period is determined, including:

[0110] For each moment within the set time period, the state estimate at that moment is determined based on the correction coefficient corresponding to the moment, the instantaneous feedback information corresponding to the moment, the required first target parameter, and the next estimated value.

[0111] Each moment within the set time period can be each moment from tn to t, where the moment corresponding to t is the current moment, and the moments t-1 to tn before t are historical moments. The value of tn can be set according to actual needs, and the present invention does not limit this.

[0112] For each moment within the set duration, the state estimate at that moment is determined based on the correction coefficient corresponding to that moment, the instantaneous feedback information corresponding to that moment, the required first target parameter, and the next estimated value. Specifically, it can be expressed as:

[0113]

[0114]

[0115] …

[0116]

[0117] In the above formula, g t-n to g t Respectively represent the state estimation value at each moment from time tn to time t; β t-n to β t Respectively represent the correction coefficients at each moment from time tn to time t; r t-n to r t They represent the instantaneous return information at each moment from time tn to time t; Q'(s t+1 ,μ'(st+1 |θ μ ')|θ Q ') represents the next estimated value; the required first target parameter may include γ, which may refer to the discounted rate of return, with a value range of [0,1]. The specific value may be set according to actual needs.

[0118] For each moment within the set time, the state estimation value of the moment is determined based on the correction coefficient corresponding to the moment, the immediate feedback information corresponding to the moment, the required first target parameter and the next estimation value, so as to facilitate the determination of the target estimation value of the next environmental state information through multiple moments within the set time.

[0119] S133. Determine a target estimated value of the next environmental state information based on the weighted average of each state estimated value.

[0120] Based on the weighted average of each state estimation value, the target estimation value of the next environmental state information is determined. The calculation formula of the target estimation value is:

[0121]

[0122] Among them, Q' new (s t+1 ,μ'(s t+1 |θ μ ')|θ Q ')) represents the target estimated value of the next environmental state information; g i It can refer to the state estimate at time i, where i can be any time from time tn to time t; Can refer to the modified weight.

[0123] Based on the weighted average of each state estimation value, the target estimation value of the next environmental state information is determined. When determining the target estimation value, the state estimation values ​​at multiple moments can be combined to make the target estimation value more accurate.

[0124] In one embodiment, after determining the target estimated value of the next environmental state information based on the weighted average of each state estimated value, the current target value y can also be calculated. t , current target value y t It can refer to the target value at the current moment determined based on the target estimate, the current instant return information and the discounted rate of return, which can be used to combine the current state value Q(s) corresponding to the current moment t ,a|θ Q ) Update the network parameters. The current target value y t The calculation formula for TD target value is:

[0125] y t =r t +γ×Q'new (s t+1 ,μ'(s t+1 |θ μ ')|θ Q '))

[0126] S134. Based on the target estimated value and the required second target parameter, the network parameters are determined, and the operation of obtaining the current environment state information is returned to continue training the traffic congestion time prediction model to be trained until the end condition is met, thereby obtaining the trained traffic congestion time prediction model.

[0127] The network parameters may refer to a set of parameters of the first network and the second network. The required second target parameters may refer to parameters required when determining the network parameters, such as μ(s|θ μ )’s learning rate α μ 、The second network Q(s,a|θ Q )’s learning rate α Q Or parameters such as the update coefficient τ of the Target network.

[0128] Determining network parameters can be achieved by following the steps below:

[0129] (1) Update the second network parameters:

[0130]

[0131]

[0132] Among them, Q(s i ,a i |θ Q ) is the state value at time i, y i is the target value at time i, N is the number of samples in the training set of this round; δ t is the mean of the TD target values ​​of N samples in the current round; α Q is the second network Q(s,a|θ Q )’s learning rate; Q(s t ,a t |θ Q ) is the current state value.

[0133] (2) Update the first network parameters:

[0134]

[0135]

[0136] Among them, α μ is the first network μ(s|θ μ )’s learning rate.

[0137] (3) Update Target network parameters:

[0138] θ Q '=τθ Q +(1-τ)θ Q '

[0139] θ μ '=τθ μ +(1-τ)θ μ '

[0140] Among them, τ is the update coefficient of the Target network.

[0141] Determining the network parameters based on the target estimate and the required second target parameters can facilitate subsequent training to train the traffic congestion time prediction model using the network parameters determined based on the target estimate and the required second target parameters.

[0142] Return to the operation of obtaining the current environmental state information to continue training the traffic congestion time prediction model to be trained until the end condition is met, and obtain the trained traffic congestion time prediction model, wherein the end condition can refer to the condition that ends the model training process, and there is no limit on the end condition, such as the training round M reaching the set training round threshold, wherein there is no limit on the set training round threshold, and it can be set according to the actual application scenario.

[0143] In one embodiment, before training the traffic congestion time prediction model to be trained, it also includes randomly initializing μ(s|θ μ ), μ'(s|θ μ '),Q(s,a|θ Q ) and Q'(s,a'|θ Q ') network parameters; at the same time, initialize the training round M, the number of iterations per round T, μ(s|θ μ ) The learning rate of the network is α μ 、Q(s,a|θ Q ) The learning rate α of the network Q , the update coefficient τ of the Target network, the initialization of the experience buffer Buffer and the corresponding buffer upper limit K (i.e., setting the length threshold), and the initialization of the correction weight

[0144] It should be noted that the process of determining network parameters based on the target estimate and the required second target parameter in steps S110 to S134 of the present embodiment is an iterative step for each round of training. In practical applications, the above training process can be continuously repeated as needed until the termination condition is met, thereby obtaining a trained traffic congestion time prediction model. The current environmental state information input into the traffic congestion time prediction model to be trained during each training session can be obtained according to practical application needs. For example, different current environmental state information can be input each time, or the same current environmental state information can be input.

[0145] The technical solution of the embodiment of the present invention is to determine the target estimated value of the next environmental state information by weightedly averaging the multiple state estimated values ​​at the current moment and multiple historical moments before the current moment corresponding to the current environmental state information in each training round, so that the target estimated value is more accurate. At the same time, the traffic congestion time prediction model to be trained can be trained through multiple operations, so that the finally trained traffic congestion time prediction model is more in line with actual needs, thereby improving the credibility of the traffic congestion time prediction model training.

[0146] The technical solution of the embodiment of the present invention, the method of determining the current correction coefficient in step S131, can more reasonably estimate the next estimated value, and guide the traffic congestion time prediction model to make reasonable decisions in a rapidly changing environment; step S132 can optimize the target estimated value of the next environmental state information based on the reward value of multiple stages, making the traffic congestion time prediction model more stable and improving the robustness of the traffic congestion time prediction model; step S133 takes into account the different weights of the state estimated values ​​at different times, so that the weight of the state estimated value farther in time is smaller, and the weight of the state estimated value closer in time is larger, making the target estimated value of the next environmental state information more reasonable. Through the above-mentioned multi-step improvement and optimization process, the traffic congestion time prediction model can be helped to reasonably estimate the target estimated value, thereby guiding the model to output a reasonable traffic congestion time (i.e., processing strategy), and improving the stability and reliability of the traffic congestion time prediction model in the traffic congestion time prediction process.

[0147] Example 3

[0148] Figure 4This is a flowchart of a driving parameter prediction method provided according to the third embodiment of the present invention. This embodiment is applicable to the actual application of a trained traffic congestion time prediction model. The method can be executed by a driving parameter prediction device, which can be implemented in the form of software and / or hardware and integrated into an electronic device. Furthermore, the electronic device includes but is not limited to: computers, laptops, smart phones, servers, etc. The electronic device can be the same electronic device as the electronic device mentioned in the first embodiment of the present invention, or it can be a different electronic device.

[0149] like Figure 4 As shown, the method includes:

[0150] S210 , inputting the environmental state information into the trained traffic congestion time prediction model to obtain action information and the predicted traffic congestion time, wherein the traffic congestion time prediction model is trained based on the traffic congestion time prediction method.

[0151] The trained traffic congestion time prediction model may refer to a traffic congestion time prediction model obtained by training using the traffic congestion time prediction method of the present invention.

[0152] The method of inputting environmental state information into the trained traffic congestion time prediction model to obtain action information and the predicted traffic congestion time is not limited, as long as the action information and the predicted traffic congestion time can be obtained. For example, an electronic device can input environmental state information into the Actor network (i.e., the first network in the present invention) included in the trained traffic congestion time prediction model through a human-computer interaction device of the electronic device to obtain action information, wherein the human-computer interaction device can display an interface for inputting environmental state information; the action information output by the Actor network is then expanded into a 30*1 tensor and spliced ​​with the state tensor to obtain a 30*(E+1) tensor, which is then input into the Critic network (i.e., the second network in the present invention) to obtain the predicted traffic congestion time.

[0153] S220: Determine a processing strategy based on the action information.

[0154] The processing strategy may refer to a strategy for determining an operation means based on the action information, wherein the operation means may be changing the vehicle speed.

[0155] The method for determining a processing strategy based on the action information is not limited, as long as the processing strategy can be determined based on the action information. For example, the processing strategy can be determined by comparing the action information with the agent's own characteristic information. The agent's own characteristic information may be the vehicle's current speed. If the speed indicated by the action information is greater than the vehicle's current speed, the vehicle's speed can be increased through electronic control.

[0156] Among them, driving parameters include action information, traffic congestion time and processing strategies.

[0157] The technical solution of the embodiment of the present invention calculates action information and traffic congestion time through a trained traffic congestion time prediction model combined with environmental status information, and determines a processing strategy based on the action information. The trained traffic congestion time prediction model can be used in practical applications to select a more appropriate processing strategy.

[0158] Example 4

[0159] Figure 5 1 is a schematic diagram of a traffic congestion time prediction device according to a fourth embodiment of the present invention. This embodiment is applicable to the case of training a traffic congestion time prediction model. Figure 5 As shown, the specific structure of the device includes:

[0160] Acquisition module 21, used to obtain current environment status information;

[0161] A first determining module 22 is used to determine the current immediate report information corresponding to the current environmental status information;

[0162] The training module 23 is used to train the traffic congestion time prediction model to be trained based on the current environmental state information and the current immediate feedback information to obtain a trained traffic congestion time prediction model. The trained traffic congestion time prediction model is used to predict traffic congestion time.

[0163] The traffic congestion time prediction device provided in this embodiment first obtains current environmental status information through an acquisition module; then determines the current instant report information corresponding to the current environmental status information through a first determination module; finally, the training module trains the traffic congestion time prediction model to be trained based on the current environmental status information and the current instant report information to obtain a trained traffic congestion time prediction model, and the trained traffic congestion time prediction model is used to predict traffic congestion time.

[0164] Furthermore, the first determining module 22 is specifically configured to:

[0165] Inputting current environment state information into a first network included in a traffic congestion time prediction model to be trained to determine current action information;

[0166] Execute the current action information and obtain the current immediate feedback information corresponding to the current environment state information.

[0167] Furthermore, the training module 23 is specifically used to:

[0168] Determine the current correction coefficient based on the current environmental status information and the current instant feedback information;

[0169] Determine the estimated state value for each moment within a set duration based on the current correction coefficient, the current instantaneous feedback information, historical information, the required first target parameter, and the next estimated value. The moments within the set duration include the current moment corresponding to the current environmental state information and the historical moments before the current moment. The historical information includes the instantaneous feedback information corresponding to the historical moments and the correction coefficients corresponding to the historical moments.

[0170] Determine a target estimated value of the next environmental state information based on a weighted average of the estimated values ​​of each state;

[0171] Based on the target estimate and the required second target parameters, the network parameters are determined, and the operation of obtaining the current environment state information is returned to continue training the traffic congestion time prediction model to be trained until the end condition is met, thereby obtaining the trained traffic congestion time prediction model.

[0172] Furthermore, the training module 23 is specifically used to:

[0173] executing current action information determined by a first network included in the traffic congestion time prediction model to be trained to obtain next environment state information;

[0174] Inputting the current environment state information and the current action information into a second network included in the traffic congestion time prediction model to be trained to obtain a current estimated value of the current environment state information;

[0175] Inputting the next environmental state information and the next action information corresponding to the next environmental state information into a second network included in the traffic congestion time prediction model to be trained to obtain a next estimated value of the next environmental state information;

[0176] A current correction factor in the correction factor is determined based on the current estimate, the next estimate, and the current instantaneous return information.

[0177] Furthermore, the training module 23 is specifically used to:

[0178] Determine the difference between the current estimate and the next estimate;

[0179] The ratio of the current instant return information to the difference is determined as the current correction coefficient in the correction coefficient.

[0180] Furthermore, the training module 23 is specifically used to:

[0181] For each moment within the set time period, the state estimate at that moment is determined based on the correction coefficient corresponding to the moment, the instantaneous feedback information corresponding to the moment, the required first target parameter, and the next estimated value.

[0182] The traffic congestion time prediction device provided in the embodiment of the present invention can execute the traffic congestion time prediction method provided in either embodiment 1 or embodiment 2 of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0183] Example 5

[0184] Figure 6 3 is a structural diagram of a driving parameter prediction device provided according to the fifth embodiment of the present invention. This embodiment is applicable to the actual application of the trained traffic congestion time prediction model.

[0185] like Figure 6 As shown, the specific structure of the device includes:

[0186] An input module 31 is configured to input environmental state information into a trained traffic congestion time prediction model to obtain action information and predicted traffic congestion time, wherein the traffic congestion time prediction model is trained based on a traffic congestion time prediction method;

[0187] A second determining module 32, configured to determine a processing strategy based on the action information;

[0188] Among them, driving parameters include action information, traffic congestion time and processing strategies.

[0189] The driving parameter prediction device provided in this embodiment first inputs environmental state information into a trained traffic congestion time prediction model through an input module to obtain action information and predicted traffic congestion time, and the traffic congestion time prediction model is trained based on a traffic congestion time prediction method; then, a processing strategy is determined based on the action information through a second determination module; wherein the driving parameters include action information, traffic congestion time and processing strategy.

[0190] The driving parameter prediction device provided in the embodiment of the present invention can execute the driving parameter prediction method provided in the third embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0191] Example 6

[0192] Figure 7A schematic diagram of the structure of an electronic device that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0193] like Figure 7 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11. The memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. Various programs and data required for the operation of the electronic device 10 can also be stored in the RAM 13. The processor 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0194] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0195] The processor 11 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any other suitable processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the traffic congestion time prediction method or the driving parameter prediction method.

[0196] In some embodiments, the traffic congestion time prediction method or the driving parameter prediction method can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the traffic congestion time prediction method or the driving parameter prediction method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to execute the traffic congestion time prediction method or the driving parameter prediction method by any other appropriate means (for example, by means of firmware).

[0197] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0198] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0199] In the context of the present invention, computer-readable storage media can be tangible media that can contain or store a computer program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Computer-readable storage media can include but are not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, computer-readable storage media can be machine-readable signal media. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0200] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0201] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0202] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0203] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0204] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A traffic congestion time prediction method, characterized in that: include: Get current environment status information; Determining current instant feedback information corresponding to the current environmental status information; Training a traffic congestion time prediction model to be trained based on the current environment state information and the current instant feedback information to obtain a trained traffic congestion time prediction model, wherein the trained traffic congestion time prediction model is used to predict traffic congestion time; The method of training the traffic congestion time prediction model to be trained based on the current environmental state information and the current immediate feedback information to obtain the trained traffic congestion time prediction model includes: Determining a current correction coefficient based on the current environmental status information and the current instant feedback information; Determine a state estimate value for each moment within a set duration based on the current correction coefficient, the current instantaneous feedback information, historical information, the required first target parameter, and the next estimated value, wherein the moments within the set duration include the current moment corresponding to the current environmental state information and historical moments before the current moment, and the historical information includes the instantaneous feedback information corresponding to the historical moments and the correction coefficients corresponding to the historical moments; Determining a target estimated value of next environmental state information based on a weighted average of the state estimated values; Based on the target estimated value and the required second target parameter, the network parameters are determined, and the operation of obtaining the current environmental state information is returned to continue training the traffic congestion time prediction model to be trained until the end condition is met, thereby obtaining the trained traffic congestion time prediction model.

2. The method according to claim 1, characterized in that The determining of the current immediate reporting information corresponding to the current environmental status information includes: Inputting the current environment state information into the first network included in the traffic congestion time prediction model to be trained to determine current action information; Execute the current action information to obtain current immediate feedback information corresponding to the current environmental state information.

3. The method according to claim 1, characterized in that The determining of the current correction coefficient based on the current environmental status information and the current instant feedback information includes: executing the current action information determined by the first network included in the traffic congestion time prediction model to be trained to obtain next environment state information; Inputting the current environment state information and the current action information into a second network included in the traffic congestion time prediction model to be trained to obtain a current estimated value of the current environment state information; Inputting the next environmental state information and the next action information corresponding to the next environmental state information into the second network included in the traffic congestion time prediction model to be trained to obtain a next estimated value of the next environmental state information; A current correction factor in the correction factors is determined based on the current estimated value, the next estimated value, and the current instant reward information.

4. The method according to claim 3, characterized in that The determining, based on the current estimated value, the next estimated value, and the current instant return information, a current correction coefficient in the correction coefficients includes: determining a difference between the current estimate and the next estimate; The ratio of the current instant return information to the difference is determined as the current correction coefficient in the correction coefficient.

5. The method according to claim 1, wherein The determining of the state estimation value at each moment within a set time period based on the current correction coefficient, the current instantaneous feedback information, the historical information, the required first target parameter, and the next estimation value includes: For each moment within the set time period, the state estimation value at the moment is determined based on the correction coefficient corresponding to the moment, the instantaneous feedback information corresponding to the moment, the required first target parameter and the next estimation value.

6. A driving parameter prediction method, characterized in that: include: Inputting environmental state information into a trained traffic congestion time prediction model to obtain action information and predicted traffic congestion time, wherein the traffic congestion time prediction model is trained based on the method according to any one of claims 1 to 5; determining a processing strategy based on the action information; The driving parameters include the action information, the traffic congestion time and the processing strategy.

7. A traffic congestion time prediction device, characterized in that: include: Acquisition module, used to obtain current environment status information; A first determining module is used to determine the current instant report information corresponding to the current environmental status information; a training module, configured to train a traffic congestion time prediction model to be trained based on the current environmental state information and the current immediate feedback information to obtain a trained traffic congestion time prediction model, wherein the trained traffic congestion time prediction model is used to predict traffic congestion time; The training module is specifically used for: Determining a current correction coefficient based on the current environmental status information and the current instant feedback information; Determine a state estimate value for each moment within a set duration based on the current correction coefficient, the current instantaneous feedback information, historical information, the required first target parameter, and the next estimated value, wherein the moments within the set duration include the current moment corresponding to the current environmental state information and historical moments before the current moment, and the historical information includes the instantaneous feedback information corresponding to the historical moments and the correction coefficients corresponding to the historical moments; Determining a target estimated value of next environmental state information based on a weighted average of the state estimated values; Based on the target estimated value and the required second target parameter, the network parameters are determined, and the operation of obtaining the current environmental state information is returned to continue training the traffic congestion time prediction model to be trained until the end condition is met, thereby obtaining the trained traffic congestion time prediction model.

8. A driving parameter prediction device, characterized in that: include: An input module, configured to input environmental state information into a trained traffic congestion time prediction model to obtain action information and predicted traffic congestion time, wherein the traffic congestion time prediction model is trained based on the method according to any one of claims 1 to 5; a second determining module, configured to determine a processing strategy based on the action information; The driving parameters include the action information, the traffic congestion time and the processing strategy.

9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5, or the method according to claim 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 5 or the method according to claim 6 is implemented.

Images