A vehicle takeover time prediction method, device, apparatus and storage medium
By combining the improved dual-depth Q-network with state prediction and estimator selection modules and delayed update modules, the takeover time prediction method solves the problem of low prediction accuracy on curved road sections, achieves accurate and fast takeover time prediction, and ensures driver safety.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- IMOTION AUTOMOTIVE TECH (SUZHOU) CO LTD
- Filing Date
- 2022-12-28
- Publication Date
- 2026-07-31
AI Technical Summary
Existing vehicle takeover time prediction schemes have low accuracy and large deviations on curved road sections, resulting in an inability to effectively guarantee driver safety.
An improved dual-deep Q-network is adopted to predict the next state through a state prediction model. Combined with the estimator selection module and delayed update module in the improved dual-deep Q-network, the takeover time is predicted, avoiding excessive exploration, improving prediction accuracy and limiting algorithm complexity.
Accurate and rapid takeover time prediction was achieved on curved road sections, ensuring driving safety and improving the model's operating efficiency and prediction accuracy.
Smart Images
Figure CN116331250B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent driving technology, and in particular to a method, apparatus, device, and storage medium for predicting vehicle takeover time. Background Technology
[0002] Intelligent connected vehicles (ICVs) are an organic combination of vehicle-to-everything (V2X) technology and intelligent vehicles. Based on intelligent driving technology, they allow drivers to perform non-driving tasks without constantly controlling the vehicle. However, currently, ICVs cannot operate under all conditions because real-world traffic scenarios present numerous complex situations that the autonomous driving systems of ICVs cannot make decisions in, leading to safety issues. To ensure the driving safety of ICVs, drivers need to take over the vehicle within a certain timeframe. Due to the high vehicle speeds on highways and the complex conditions on curved roads, especially ramps (curves), drivers need to quickly and accurately take over the vehicle. This depends on the prediction accuracy and time complexity of the vehicle takeover time prediction model. Existing takeover time prediction schemes have low accuracy and large deviations on curved roads, making it difficult to effectively guarantee driver safety.
[0003] Therefore, the aforementioned technical problems urgently need to be solved by those skilled in the art. Summary of the Invention
[0004] In view of this, the purpose of this invention is to provide a method, apparatus, device, and storage medium for predicting vehicle take-off time, which can accurately and quickly predict take-off time on curved roads, ensuring driving safety. The specific solution is as follows:
[0005] The first aspect of this application provides a method for predicting vehicle takeover time, including:
[0006] Obtain target driving information of the target vehicle on a curved road section;
[0007] The target driving information is input into the state prediction model, so as to output the next state of the target vehicle using the state prediction model;
[0008] The target driving information and the next state are simultaneously input into the takeover time prediction model constructed based on the improved dual-depth Q network, so as to predict the target takeover time of the target vehicle using the takeover time prediction model; wherein, the improved dual-depth Q network has an estimator selection module and a delayed update module; the estimator selection module is used to select the optimal estimator, and the delayed update module is used to indicate that the estimator parameters will not be updated in this round of update.
[0009] Optionally, the target driving information includes its own vehicle operation information, road information, surrounding vehicle operation information, and driver status information.
[0010] Optionally, before inputting the target driving information into the state prediction model, the method further includes:
[0011] The target driving information data format is converted into a first data format supported by the input state prediction model, and the target driving information in the first data format is input into the state prediction model.
[0012] Optionally, before simultaneously inputting the target driving information and the next state into the takeover time prediction model constructed based on the improved dual-depth Q-network, the method further includes:
[0013] The target driving information is converted into a second data format supported by the takeover time prediction model, and the target driving information in the second data format is input into the takeover time prediction model.
[0014] Optionally, predicting the target takeover time of the target vehicle using the takeover time prediction model includes:
[0015] The state action value is calculated using the two estimators in the aforementioned takeover time prediction model;
[0016] The estimator selection module in the takeover time prediction model selects a target estimator from two estimators based on a preset probability factor, and determines the optimal action and executes the optimal action based on the state action value corresponding to the target estimator; wherein, the preset probability factor is a predefined hyperparameter, and its positive and negative deviations are adjusted according to the road scenario in each update.
[0017] The delayed update module in the takeover time prediction model determines whether to update the estimator parameters in this round of updates based on the execution result. If yes, the estimator parameters are updated; otherwise, the estimator parameters are not updated.
[0018] The next round of calculations is performed until the final state is reached, at which point the calculations stop, and the final state action value is used as the predicted target takeover time.
[0019] Optionally, the step of using the delayed update module in the takeover time prediction model to determine whether to update the estimator parameters in this round of updates based on the execution result includes:
[0020] Based on the first relation, determine whether the update round number corresponding to the execution result is the same as the current update round number. If it is, determine whether the estimator parameters will be updated in the current update round. If not, determine whether the estimator parameters will not be updated in the current update round.
[0021] The first relation is:
[0022]
[0023] Among them, Q t+1 (s,a) represents the execution result, k represents the update round number corresponding to the execution result, and r wi s represents the next state after the i-th execution result. wi This represents the next reward for the i-th execution result, ψ is the error coefficient, s is the current state, a is the current behavior, and γ is the model parameter.
[0024] Optionally, after the takeover time prediction model predicts the target takeover time of the target vehicle, it further includes:
[0025] The predicted target takeover time is output to the display interface of the preset device for display, and the voice broadcasting device of the preset device is controlled to broadcast the target takeover time in a preset broadcasting mode.
[0026] Optionally, the display interface of the preset device is the central control screen of the target vehicle.
[0027] A second aspect of this application provides a vehicle takeover time prediction device, comprising:
[0028] The driving information acquisition module is used to acquire the target driving information of the target vehicle on the curved road section;
[0029] The state prediction module is used to input the target driving information into the state prediction model, so as to use the state prediction model to output the next state of the target vehicle.
[0030] The takeover time prediction module is used to simultaneously input the target driving information and the next state into a takeover time prediction model constructed based on an improved dual-depth Q-network, so as to use the takeover time prediction model to predict the target takeover time of the target vehicle; wherein, the improved dual-depth Q-network has an estimator selection module and a delayed update module; the estimator selection module is used to select the optimal estimator, and the delayed update module is used to indicate that the estimator parameters will not be updated in this round of update.
[0031] A third aspect of this application provides an electronic device including a processor and a memory; wherein the memory is used to store a computer program loaded and executed by the processor to implement the aforementioned vehicle takeover time prediction method.
[0032] A fourth aspect of this application provides a computer-readable storage medium storing computer-executable instructions, which, when loaded and executed by a processor, implement the aforementioned vehicle takeover time prediction method.
[0033] In this application, target driving information of the target vehicle on a curved road section is first obtained; then, the target driving information is input into a state prediction model to output the next state of the target vehicle; finally, the target driving information and the next state are simultaneously input into a takeover time prediction model constructed based on an improved dual-depth Q-network to predict the target takeover time of the target vehicle; wherein, the improved dual-depth Q-network has an estimator selection module and a delayed update module; the estimator selection module is used to select the optimal estimator, and the delayed update module is used to indicate that the estimator parameters will not be updated in this round of update. It can be seen that before using the takeover time prediction model to predict the takeover time, this application first uses the prediction model to predict the next state of the obtained target driving information, and then inputs the next state and target driving information together into the takeover time prediction model to predict the takeover time, avoiding excessive exploration by the takeover time prediction model. Meanwhile, the takeover time prediction model adds an estimator selection module and a delayed update module to the dual-deep Q-network. In the process of predicting takeover time, the estimator selection module can balance between overestimation and underestimation, and the delayed update module can limit the algorithm complexity, thereby making accurate and fast predictions of takeover time on curved road sections and ensuring driving safety. Attached Figure Description
[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0035] Figure 1 A flowchart of a vehicle takeover time prediction method provided in this application;
[0036] Figure 2 A structural comparison diagram of a traditional DDQN network and an improved DDQN network is provided for this application;
[0037] Figure 3 A flowchart illustrating a specific vehicle takeover time prediction method provided in this application;
[0038] Figure 4This application provides a comparison chart of the performance of a traditional DDQN network and an improved DDQN network in predicting vehicle takeover time.
[0039] Figure 5 This application provides a schematic diagram of a vehicle takeover time prediction device.
[0040] Figure 6 This application provides a structural diagram of an electronic device for predicting vehicle takeover time. Detailed Implementation
[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0042] Existing takeover time prediction schemes exhibit low accuracy and significant bias on curved roads, compromising driver safety. To address these shortcomings, this application provides a vehicle takeover time prediction scheme. Before using a takeover time prediction model, the scheme first uses the model to predict the next state of the acquired target driving information. Then, the next state and target driving information are input together into the takeover time prediction model to predict the takeover time, avoiding excessive exploration by the model. Furthermore, the takeover time prediction model incorporates an estimator selection module and a delayed update module within its dual-depth Q-network. During the takeover time prediction process, the estimator selection module balances overestimation and underestimation, while the delayed update module limits algorithm complexity. This enables accurate and rapid prediction of takeover time on curved roads, ensuring driving safety.
[0043] Figure 1 A flowchart illustrating a vehicle takeover time prediction method provided in this application embodiment. See also... Figure 1 As shown, the vehicle takeover time prediction method includes:
[0044] S11: Obtain the target vehicle's driving information on the curved road section.
[0045] In this embodiment, the target vehicle's driving information on the curved road section is first obtained. The target driving information includes the vehicle's own driving information, road information, surrounding vehicle driving information, and driver status information. Among them, the vehicle's own driving information includes, but is not limited to, the vehicle's speed, acceleration, steering wheel angle, and throttle opening; the road information includes, but is not limited to, the road's curvature, width, slope, and length; the surrounding vehicle driving information includes, but is not limited to, the surrounding vehicles' positions, speeds, accelerations, steering wheel angles, and expected driving trajectories; the driver status information includes, but is not limited to, the driver's eye opening, blinking frequency, eye focus, and hand and foot placement.
[0046] In this embodiment, the above-mentioned information acquisition process can be implemented using an information acquisition module, which includes a vehicle body sensor for collecting its own vehicle operation information, a road information acquisition device for collecting road information, an intelligent connected information receiving device for collecting the operation information of surrounding vehicles, and a driver condition monitoring device for collecting driver status information.
[0047] S12: Input the target driving information into the state prediction model, and use the state prediction model to output the next state of the target vehicle.
[0048] In this embodiment, after acquiring the target driving information, the target driving information is first input into a state prediction model to output the next state of the target vehicle. The state prediction model predicts the input target driving information and combines the predicted data into the next state S'. Before inputting data into the takeover time prediction model, the state prediction model is used to predict the already collected target driving information, that is, to predict a portion of the future data first. This avoids the takeover time prediction model from performing excessive exploration and improves the model's operating efficiency. The state prediction model includes, but is not limited to, machine learning and deep learning prediction algorithms, such as linear regression, random forest, SVM, etc.
[0049] It is understood that before inputting the target driving information into the state prediction model, the data format of the target driving information needs to be converted into a first data format supported by the input state prediction model, and then the target driving information in the first data format is input into the state prediction model.
[0050] S13: The target driving information and the next state are simultaneously input into the takeover time prediction model constructed based on the improved dual-depth Q network, so as to use the takeover time prediction model to predict the target takeover time of the target vehicle; wherein, the improved dual-depth Q network has an estimator selection module and a delayed update module; the estimator selection module is used to select the optimal estimator, and the delayed update module is used to indicate that the estimator parameters will not be updated in this round of update.
[0051] In this embodiment, after obtaining the next state of the target vehicle, the target driving information and the next state can be simultaneously input into a takeover time prediction model constructed based on an improved Double Deep Q Network to predict the target takeover time of the target vehicle. The improved Double Deep Q Network has an estimator selection module and a delayed update module. Specifically, the improved Double Deep Q Network is a network obtained by adding an estimator selection module and a delayed update module to a traditional Double Deep Q Network. The estimator selection module is used to select the optimal estimator, and the delayed update module is used to indicate that the estimator parameters will not be updated in this round of updates. The Double Deep Q Network, also known as the DDQN network, is a machine learning algorithm. Similarly, before inputting the target driving information into the takeover time prediction model, the data format of the target driving information needs to be converted into a second data format supported by the takeover time prediction model, and then the target driving information in the second data format is input into the takeover time prediction model.
[0052] Considering that traditional DDQN predictions have limited practical value due to the lack of consideration for algorithm time complexity, prediction bias, and vehicle lateral movement, this embodiment improves upon the traditional DDQN network by adding an estimator selection module and a delayed update module. The estimator selection module allows for a trade-off between overestimation and underestimation, improving the model's prediction accuracy. The delayed update module limits algorithm complexity, further enhancing the model's efficiency and enabling extremely rapid prediction of intelligent connected vehicle takeover times on highway curves. Figure 2 The diagram shows a structural comparison between the traditional DDQN network and the improved DDQN network. The traditional DDQN network uses two independent estimators to approximate the Q-value to address the overestimation problem when predicting takeover time. However, overestimation / underestimation does not necessarily negatively impact the prediction results. Overestimating the reward for an action encourages the model to explore more areas, while underestimating the reward improves the model's efficiency. In this embodiment, to utilize overestimation and underestimation for more accurate takeover time prediction, the improved DDQN network incorporates an estimator selection module (also known as a tradeoff module). However, this module increases model complexity. To further improve efficiency, a delayed update module is added, employing a delayed update rule to reduce the algorithm's time complexity.
[0053] In this embodiment, after predicting the target takeover time of the target vehicle using the takeover time prediction model, the predicted target takeover time can be further output to the display interface of a preset device for display. Simultaneously, the voice broadcast device of the preset device can be controlled to broadcast the target takeover time according to a preset broadcast method. Specifically, the display interface of the preset device is the central control screen of the target vehicle. The central control screen receives and displays the target takeover time transmitted from the improved DDQN network. Simultaneously, to enable the driver to obtain information more quickly, the target takeover time transmitted from the improved DDQN network also sends a signal to the voice broadcast device. Upon receiving the signal, the voice broadcast device emits an audio prompt to the driver to take over as soon as possible.
[0054] As can be seen, this embodiment first obtains the target vehicle's driving information on a curved road section; then, it inputs the target driving information into a state prediction model to output the next state of the target vehicle; finally, it simultaneously inputs the target driving information and the next state into a takeover time prediction model constructed based on an improved dual-depth Q-network to predict the target takeover time of the target vehicle. The improved dual-depth Q-network has an estimator selection module and a delayed update module; the estimator selection module selects the optimal estimator, and the delayed update module indicates that the estimator parameters will not be updated in this round of updates. Before using the takeover time prediction model to predict the takeover time, this embodiment first uses the prediction model to predict the next state of the obtained target driving information. Based on this, the next state and the target driving information are input together into the takeover time prediction model to predict the takeover time, avoiding excessive exploration by the takeover time prediction model. Meanwhile, the takeover time prediction model adds an estimator selection module and a delayed update module to the dual-deep Q-network. In the process of predicting takeover time, the estimator selection module can balance between overestimation and underestimation, and the delayed update module can limit the algorithm complexity, thereby making accurate and fast predictions of takeover time on curved road sections and ensuring driving safety.
[0055] Figure 3 A flowchart illustrating a specific vehicle takeover time prediction method provided in this application embodiment. See also... Figure 3 As shown, the vehicle takeover time prediction method includes:
[0056] S21: Obtain the target vehicle's driving information on the curved road section.
[0057] S22: Input the target driving information into the state prediction model, and use the state prediction model to output the next state of the target vehicle.
[0058] S23: Input the target driving information and the next state into the takeover time prediction model constructed based on the improved dual-depth Q network.
[0059] In this embodiment, the specific processes of steps S21 to S23 can be referred to the corresponding contents disclosed in the foregoing embodiments, and will not be repeated here.
[0060] S24: Calculate the state action value using the two estimators in the takeover time prediction model.
[0061] S25: Using the estimator selection module in the takeover time prediction model, a target estimator is selected from two estimators based on a preset probability factor, and the optimal action is determined and executed based on the state action value corresponding to the target estimator; wherein, the preset probability factor is a predefined hyperparameter, and its positive and negative deviations are adjusted according to the road scenario in each update.
[0062] S26: The delayed update module in the takeover time prediction model determines whether to update the estimator parameters in this round of updates based on the execution result. If yes, the estimator parameters are updated; otherwise, the estimator parameters are not updated.
[0063] S27: Execute the next round of calculation until the final state is reached and stop the calculation, and use the final state action value as the predicted target takeover time.
[0064] In this embodiment, when using the takeover time prediction model built based on the improved dual-deep Q-network, the state-action value is first calculated using two estimators in the takeover time prediction model. Then, the estimator selection module in the takeover time prediction model selects a target estimator from the two estimators according to a preset probability factor, and determines the optimal action and executes the optimal action based on the state-action value corresponding to the target estimator; wherein, the preset probability factor is a predefined hyperparameter, and its positive and negative deviations are adjusted according to the road scenario in each update. Next, the delayed update module in the takeover time prediction model determines whether to update the estimator parameters in this update based on the execution result. If yes, the estimator parameters are updated; otherwise, the estimator parameters are not updated. Finally, the next round of calculation is executed until the final state is reached and the calculation stops, and the final state-action value is used as the predicted target takeover time.
[0065] Specifically, in delayed updates, the update round number corresponding to the execution result is determined based on the first relational expression to determine whether it is the current update round number. If it is, the estimator parameters are updated in this round; otherwise, the estimator parameters are not updated in this round. The first relational expression is:
[0066]
[0067] Among them, Q t+1 (s,a) represents the execution result, k represents the update round number corresponding to the execution result, and r wi s represents the next state after the i-th execution result. wi This represents the next reward for the i-th execution result, ψ is the error coefficient, s is the current state, a is the current behavior, and γ is the model parameter.
[0068] To facilitate understanding, the workflow of a traditional DDQN network is described below. The target driving information collected by the information acquisition module is input into the traditional DDQN network. After receiving the data, the traditional DDQN network initializes the number of iterations T, state feature dimension n, action set A, step size α, decay factor γ, exploration rate ∈ , current Q-network Q, target Q-network Q', number of samples for batch gradient descent m, and target Q-network parameter update frequency C. First, the values Q corresponding to all states and actions are randomly initialized. All parameters of the current Q-network are randomly initialized, and the parameters ω' = ω of the target Q-network Q' are initialized. The experience replay set D is cleared. Next, S is initialized as the first state of the current state sequence.
[0069] The first step is to obtain its feature vector φ(S) and use φ(S) as input in the Q-network to obtain the Q-value output corresponding to all actions of the Q-network. A greedy algorithm is used to select the corresponding action A from the current Q-value output. The second step is to execute the current action A based on the state S and S', obtaining the feature vector φ(S') and reward R corresponding to the new state S', and determining whether to terminate the state ifend. The quintuple {φ(S), A, R, φ(S'), ifend} is stored in the experience replay set D, and S = S'. The third step is to sample m samples {φ(Sj), Aj, Rj, φ(S'j), ifendj}, j = 1, 2...m, from the experience replay set D, and calculate the current target Q-value y. j Among them, y j The calculation formula is:
[0070]
[0071] The third step uses the mean squared error loss function to update all parameters of the Q-network through backpropagation of the neural network's gradients. If S' is a terminated state, the current iteration is complete; otherwise, return to the first step. This process is repeated for T iterations. After training and outputting the Q-network, the model can predict the takeover time based on the current model's input.
[0072] It should be noted that in the improved DDQN network, it uses mutually independent estimators QA and Q... BApproximating the Q value, in S t+1 In the state, from Q A Select the optimal action set Max A (Q(S t+1 )),Right now Randomly select action a from the optimal action set. * And retrieve the value corresponding to Q. B The element on Q A The following updates have been made:
[0073] Q A (S t A t )←Q A (S t A t )+α(R t+1 +γQ B (S t+1 ,a * )-Q A (S t A t )
[0074] Similarly, Q B The update is as follows:
[0075] Q B (S t A t )←Q B (S t A t )+α(R t+1 +γQ A (S t+1 ,a * )-Q B (S t A t )
[0076] Traditional DDQN networks first identify the action corresponding to the maximum Q-value in the current Q-network, and then use this selected action to calculate the target Q-value in the target network, thus eliminating overestimation. However, overestimation / underestimation is not necessarily detrimental. Therefore, the improved DDQN network defines the aforementioned preset probability factor. The improved DDQN network uses... The probability is based on Q A Select action and The probability is based on Q B Choose an action. As a hyperparameter, the positive and negative deviations can be adjusted according to the road scenario. To reduce the pass probability, Q is chosen. A Or Q BThe improved DDQN network addresses the complexity introduced by action A by employing a k-round delay based on the DDQN update rule. This allows the model to predict the takeover time in a short period of time, as expressed in the formula:
[0077]
[0078] In the above process, the reward R of the improved DDQN network is given by the action A performed in the current state S and the action A' performed in the next state S'.
[0079] Figure 4 This is a comparison chart of the vehicle takeover time prediction performance of a traditional DDQN network and an improved DDQN network provided in this embodiment. Time t1 is the takeover time predicted by the prediction method in this embodiment, and t2 is the takeover time predicted by the traditional DDQN network. It can be seen that this embodiment uses a method based on states S and S' to select action A and a preset probability factor. Select Q A and Q B The delayed update rule limits the algorithm complexity and the method of overestimating the lateral movement of the vehicle is used to predict the takeover time, making the prediction of the takeover time faster and more accurate. It can better solve the problem of predicting the takeover time of intelligent connected vehicles on the curved sections of highways.
[0080] See Figure 5 As shown in the illustration, this application also discloses a vehicle takeover time prediction device, comprising:
[0081] The driving information acquisition module 11 is used to acquire the target driving information of the target vehicle on the curved road section;
[0082] The state prediction module 12 is used to input the target driving information into the state prediction model, so as to use the state prediction model to output the next state of the target vehicle.
[0083] The takeover time prediction module 13 is used to simultaneously input the target driving information and the next state into a takeover time prediction model constructed based on the improved dual-depth Q network, so as to use the takeover time prediction model to predict the target takeover time of the target vehicle; wherein, the improved dual-depth Q network has an estimator selection module and a delayed update module; the estimator selection module is used to select the optimal estimator, and the delayed update module is used to indicate that the estimator parameters will not be updated in this round of update.
[0084] As can be seen, this embodiment first obtains the target vehicle's driving information on a curved road section; then, it inputs the target driving information into a state prediction model to output the next state of the target vehicle; finally, it simultaneously inputs the target driving information and the next state into a takeover time prediction model constructed based on an improved dual-depth Q-network to predict the target takeover time of the target vehicle. The improved dual-depth Q-network has an estimator selection module and a delayed update module; the estimator selection module selects the optimal estimator, and the delayed update module indicates that the estimator parameters will not be updated in this round of updates. Before using the takeover time prediction model to predict the takeover time, this embodiment first uses the prediction model to predict the next state of the obtained target driving information. Based on this, the next state and the target driving information are input together into the takeover time prediction model to predict the takeover time, avoiding excessive exploration by the takeover time prediction model. Meanwhile, the takeover time prediction model adds an estimator selection module and a delayed update module to the dual-deep Q-network. In the process of predicting takeover time, the estimator selection module can balance between overestimation and underestimation, and the delayed update module can limit the algorithm complexity, thereby making accurate and fast predictions of takeover time on curved road sections and ensuring driving safety.
[0085] In some specific embodiments, the target driving information in the vehicle takeover time prediction device includes its own vehicle operation information, road information, surrounding vehicle operation information, and driver status information.
[0086] In some specific embodiments, the vehicle takeover time prediction device further includes:
[0087] The first format conversion module is used to convert the data format of the target driving information into a first data format supported by the input state prediction model, and input the target driving information in the first data format into the state prediction model.
[0088] The second format conversion module is used to convert the data format of the target driving information into a second data format supported by the takeover time prediction model, and input the target driving information in the second data format into the takeover time prediction model;
[0089] The display and broadcast module is used to output the predicted target takeover time to the display interface of the preset device for display, and to control the voice broadcast device of the preset device to broadcast the target takeover time in a preset broadcast mode.
[0090] In some specific embodiments, the takeover time prediction module 13 specifically includes:
[0091] The calculation unit is used to calculate the state action value using two estimators in the takeover time prediction model;
[0092] The selection unit is used to select a target estimator from two estimators based on a preset probability factor using the estimator selection module in the takeover time prediction model, and to determine the optimal action and execute the optimal action based on the state action value corresponding to the target estimator; wherein, the preset probability factor is a predefined hyperparameter, and its positive and negative deviations are adjusted according to the road scenario in each update.
[0093] The update unit is used to determine whether to update the estimator parameters in this round of update based on the execution result using the delayed update module in the takeover time prediction model. If yes, the estimator parameters are updated; otherwise, the estimator parameters are not updated.
[0094] The update unit is used to perform the next round of calculations until the final state is reached and the calculations stop, and the final state action value is used as the predicted target takeover time.
[0095] In some specific embodiments, the update unit is specifically used to determine whether the update round number corresponding to the execution result is the current update round number according to the first relation. If it is, the estimator parameters are determined to be updated in the current update round; if not, the estimator parameters are determined not to be updated in the current update round.
[0096] The first relation is:
[0097]
[0098] Among them, Q t+1 (s,a) represents the execution result, k represents the update round number corresponding to the execution result, rwi represents the next state of the i-th execution result, swi represents the next reward of the i-th execution result, ψ is the error coefficient, s is the current state, a is the current behavior, and γ is the model parameter.
[0099] Furthermore, embodiments of this application also provide an electronic device. Figure 6 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.
[0100] Figure 6This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the vehicle takeover time prediction method disclosed in any of the foregoing embodiments.
[0101] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.
[0102] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222 and data 223, etc., and the storage method can be temporary storage or permanent storage.
[0103] The operating system 221 manages and controls the various hardware devices and computer programs 222 on the electronic device 20 to enable the processor 21 to perform calculations and processing on the massive amounts of data 223 in the memory 22. It can be Windows Server, Netware, Unix, Linux, etc. The computer program 222, in addition to including a computer program capable of performing the vehicle takeover time prediction method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, may further include computer programs capable of performing other specific tasks. The data 223 may include target driving information collected by the electronic device 20.
[0104] Furthermore, this application also discloses a storage medium storing a computer program. When the computer program is loaded and executed by a processor, it implements the vehicle takeover time prediction method steps disclosed in any of the foregoing embodiments.
[0105] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.
[0106] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0107] The vehicle takeover time prediction method, apparatus, device, and storage medium provided by the present invention have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.
Claims
1. A method for predicting vehicle takeover time, characterized in that, include: Obtain target driving information of the target vehicle on a curved road section; The target driving information is input into the state prediction model, and the state prediction model is used to output the next state of the target vehicle. The target driving information and the next state are simultaneously input into the takeover time prediction model constructed based on the improved dual-depth Q network, so as to use the takeover time prediction model to predict the target takeover time of the target vehicle; wherein, the improved dual-depth Q network has an estimator selection module and a delayed update module; the estimator selection module is used to select the optimal estimator, and the delayed update module is used to indicate that the estimator parameters will not be updated in this round of update; The process of predicting the target takeover time of the target vehicle using the takeover time prediction model includes: The delayed update module in the takeover time prediction model determines whether to update the estimator parameters in this round of updates based on the execution result. If yes, the estimator parameters are updated; otherwise, the estimator parameters are not updated. The step of using the delayed update module in the takeover time prediction model to determine whether to update the estimator parameters in this round of updates based on the execution result includes: Based on the first relation, determine whether the update round number corresponding to the execution result is the same as the current update round number. If it is, determine whether the estimator parameters will be updated in the current update round. If not, determine whether the estimator parameters will not be updated in the current update round. The first relation is: in, Indicates the execution result. This indicates the update round number corresponding to the execution result. This indicates the next state after the i-th execution result. This indicates the next reward for the i-th execution result. The error coefficient, This is the current state. For the current action, These are the model parameters.
2. The vehicle takeover time prediction method according to claim 1, characterized in that, The target driving information includes its own vehicle operation information, road information, surrounding vehicle operation information, and driver status information.
3. The vehicle takeover time prediction method according to claim 1, characterized in that, Before inputting the target driving information into the state prediction model, the method further includes: The target driving information data format is converted into a first data format supported by the input state prediction model, and the target driving information in the first data format is input into the state prediction model. Before simultaneously inputting the target driving information and the next state into the takeover time prediction model constructed based on the improved dual-depth Q-network, the method further includes: The target driving information is converted into a second data format supported by the takeover time prediction model, and the target driving information in the second data format is input into the takeover time prediction model.
4. The vehicle takeover time prediction method according to claim 1, characterized in that, Before the delayed update module in the takeover time prediction model determines whether to update the estimator parameters in this round of updates based on the execution result, the method further includes: The state action value is calculated using the two estimators in the aforementioned takeover time prediction model; The estimator selection module in the takeover time prediction model selects a target estimator from two estimators based on a preset probability factor, and determines the optimal action and executes the optimal action based on the state action value corresponding to the target estimator; wherein, the preset probability factor is a predefined hyperparameter, and its positive and negative deviations are adjusted according to the road scenario in each update. If not, then after not updating the estimator parameters, the method further includes: The next round of calculations is performed until the final state is reached, at which point the calculations stop, and the final state action value is used as the predicted target takeover time.
5. The vehicle takeover time prediction method according to any one of claims 1 to 4, characterized in that, After the takeover time prediction model predicts the target takeover time of the target vehicle, it also includes: The predicted target takeover time is output to the display interface of the preset device for display, and the voice broadcasting device of the preset device is controlled to broadcast the target takeover time in a preset broadcasting mode.
6. The vehicle takeover time prediction method according to claim 5, characterized in that, The display interface of the preset device is the central control screen of the target vehicle.
7. A vehicle takeover time prediction device, characterized in that, include: The driving information acquisition module is used to acquire the target driving information of the target vehicle on the curved road section; The state prediction module is used to input the target driving information into the state prediction model, so as to use the state prediction model to output the next state of the target vehicle. The takeover time prediction module is used to simultaneously input the target driving information and the next state into a takeover time prediction model constructed based on an improved dual-depth Q-network, so as to use the takeover time prediction model to predict the target takeover time of the target vehicle; wherein, the improved dual-depth Q-network has an estimator selection module and a delayed update module; the estimator selection module is used to select the optimal estimator, and the delayed update module is used to indicate that the estimator parameters will not be updated in this round of update; The takeover time prediction module includes: The update unit is used to determine whether to update the estimator parameters in this round of update based on the execution result using the delayed update module in the takeover time prediction model. If yes, the estimator parameters are updated; otherwise, the estimator parameters are not updated. The update unit is specifically used to: determine whether the update round number corresponding to the execution result is the current update round number according to the first relation; if so, determine that the estimator parameters will be updated in the current update round; if not, determine that the estimator parameters will not be updated in the current update round. The first relation is: in, Indicates the execution result. This indicates the update round number corresponding to the execution result. This indicates the next state after the i-th execution result. This indicates the next reward for the i-th execution result. The error coefficient, This is the current state. For the current action, These are the model parameters.
8. An electronic device, characterized in that, The electronic device includes a processor and a memory; wherein the memory is used to store a computer program, which is loaded and executed by the processor to implement the vehicle takeover time prediction method as described in any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that, Used to store computer-executable instructions, which, when loaded and executed by a processor, implement the vehicle takeover time prediction method as described in any one of claims 1 to 6.