Vehicle process management method, vehicle, and storage medium
By optimizing process priority configuration through deep reinforcement learning, the problem of inaccurate manual configuration is solved, and the automatic adjustment and real-time update of process priority are realized, thereby improving the response efficiency of the vehicle system and the user experience.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- GUANGZHOU XIAOPENG MOTORS TECH CO LTD
- Filing Date
- 2022-08-19
- Publication Date
- 2026-06-09
Smart Images
Figure CN115373817B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of intelligent vehicle technology, and in particular to a vehicle process management method, a vehicle, and a storage medium. Background Technology
[0002] In real-world driving scenarios, the Autonomous Driving Intelligent Control Unit (XPU) runs multiple autonomous driving processes in the background. These processes are stored in a ready queue, and the processor allocates computing resources based on their priorities, with higher-priority processes processed first and lower-priority processes processed later. To improve the overall response time of all processes, the priority of each application (represented by a nice value in Linux systems) can be adjusted to achieve optimal overall priority matching. This results in faster system response times when switching scenarios, providing users with a better driving experience.
[0003] Currently, the nice value of background processes is mainly adjusted manually. However, this manual method is static, cannot be changed, and it's impossible to know if the final result is optimal; it only allows for repeated trials. In other words, in real-world applications, directly obtaining the optimal process priority configuration is very difficult, and relying solely on manual configuration is not accurate enough. Summary of the Invention
[0004] This application provides a vehicle process management method, a vehicle, and a storage medium.
[0005] This application provides a vehicle process management method. The vehicle process management method includes: repeatedly acquiring the current state of multiple process priority combinations running in a vehicle driving scenario, the reward value of the current process priority combination, the action of configuring priorities, and the next state of the process priority combinations after configuring priorities according to the action, and adding them as experience sequences to an experience replay pool; selecting a preset number of the experience sequences from the experience replay pool and processing them to obtain a target priority combination; and managing the multiple processes running in the corresponding vehicle driving scenario according to the target priority combination.
[0006] In some implementations, the step of repeatedly acquiring the current state of multiple process priority combinations running in a vehicle driving scenario, the reward value of the current process priority combination, the action of configuring the priority, and the next state of the process priority combination after configuring the priority according to the action, and adding them as an experience sequence to the experience replay pool includes: acquiring the original reward value of each current state; outputting the Q value of each current state through a value network; and determining the reward value based on the original reward value and the Q value if the next state is not the final state.
[0007] In some implementations, obtaining the original reward value for each current state includes: obtaining the total response time of the multiple processes corresponding to each current state; if the total response time is greater than a preset time threshold, then determining the original reward value as a first set value; or if the total response time is not greater than the preset time threshold, then determining the original reward value as a second set value.
[0008] In some implementations, determining the reward value based on the original reward value and the Q value when the next state is not the final state includes: adding a discount factor to the original reward value and the product of the maximum Q value that has been output to obtain the reward value.
[0009] In some implementations, the process of repeatedly acquiring the current state of multiple process priority combinations running in a vehicle driving scenario, the reward value of the current process priority combination, the action of configuring priorities, and the next state of the process priority combination after configuring priorities according to the action, and adding them to the experience replay pool as an experience sequence, includes: acquiring the action of configuring priorities each time using a greedy strategy, wherein the greedy value of the greedy strategy is inversely proportional to the number of acquisitions.
[0010] In some implementations, the step of selecting a preset number of experience sequences from the experience replay pool for processing to obtain a target process priority combination includes: selecting a preset number of experience sequences from the experience replay pool to train a deep reinforcement learning model; calculating the loss value of the deep reinforcement learning model; updating the parameters of the deep reinforcement learning model based on the loss value; and determining the target priority combination based on the updated deep reinforcement learning model.
[0011] In some implementations, calculating the loss value of the deep reinforcement learning model includes: outputting the Q-value of each current state through the deep reinforcement learning model; and calculating the loss value of the deep reinforcement learning model based on the reward value and the corresponding Q-value.
[0012] In some implementations, determining the target priority combination based on the updated deep reinforcement learning model includes: if the number of updates to the deep reinforcement learning model reaches a preset number, then determining that the deep reinforcement learning model update is complete; and determining the target priority combination based on the updated deep reinforcement learning model.
[0013] This application provides a vehicle. The vehicle includes a processor and a memory, the memory storing a computer program that, when executed by the processor, implements the vehicle process management method described in any of the above embodiments.
[0014] This application also provides a non-volatile computer-readable storage medium containing a computer program. When the computer program is executed by one or more processors, it implements the vehicle process management method described in any of the above embodiments.
[0015] This application establishes an experience replay pool by combining the state of process priority combinations, the corresponding reward value, and the action of configuring process priorities. It then processes the data in the experience replay pool to find the optimal priority configuration, i.e., the target priority combination, and manages multiple processes running in the corresponding vehicle driving scenario based on this target priority combination. This eliminates the need for manual configuration and makes the priority configuration of multiple processes more accurate.
[0016] Additional aspects and advantages of the embodiments of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description
[0017] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, wherein:
[0018] Figure 1 This is a flowchart illustrating the vehicle process management method of this application;
[0019] Figure 2 This is a schematic diagram of the Markov decision process in the DQN-PPO algorithm of this application;
[0020] Figure 3 This is a schematic diagram of the process priority configuration in this application;
[0021] Figure 4 This is a flowchart illustrating the vehicle process management method of this application;
[0022] Figure 5 This is a flowchart illustrating the vehicle process management method of this application;
[0023] Figure 6 This is a flowchart illustrating the vehicle process management method of this application;
[0024] Figure 7 This is a flowchart illustrating the vehicle process management method of this application;
[0025] Figure 8 This is a flowchart illustrating the vehicle process management method of this application;
[0026] Figure 9 This is a flowchart illustrating the vehicle process management method of this application. Detailed Implementation
[0027] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the embodiments of this application, and should not be construed as limiting the embodiments of this application.
[0028] Currently, the nice value of background processes is mainly adjusted manually. However, this manual method is static, cannot be changed, and it's impossible to know if the final result is optimal; it can only be done through repeated trials. In other words, in real-world applications, directly obtaining the optimal process priority configuration is very difficult. Relying solely on manual configuration is not only inaccurate but also cannot be updated in real time according to different driving scenarios.
[0029] In view of this, please refer to Figure 1 This application provides a vehicle process management method. The vehicle process management method includes:
[0030] 01: The current state of multiple process priority combinations running in the vehicle driving scenario, the reward value of the current process priority combination, the action of configuring the priority, and the next state of the process priority combination after configuring the priority according to the action are obtained multiple times and added to the experience replay pool as an experience sequence.
[0031] 02: Select a preset number of experience sequences from the experience replay pool and process them to obtain the target priority combination;
[0032] 03: Manage multiple processes running in the corresponding vehicle driving scenario based on target priority.
[0033] This application also provides a vehicle. The vehicle includes a processor and a memory, on which a computer program is stored. The processor is used to repeatedly acquire the current state of multiple process priority combinations running in a vehicle driving scenario, the reward value of the current process priority combination, the action of configuring priorities, and the next state of the process priority combination after configuring priorities according to the action, as an experience sequence and add them to an experience replay pool; select a preset number of experience sequences from the experience replay pool for processing to obtain a target priority combination; and manage multiple processes running in the corresponding vehicle driving scenario according to the target priority combination.
[0034] Understandably, XPU possesses extremely high computing power, enabling it to easily train a deep learning model. This application proposes a Process Priority Optimization Method Based on Deep Reinforcement Learning (DQN-PPO) to adaptively learn the optimal priority combination for each vehicle driving scenario (such as automatic parking across floors, reversing into a parking space, high-speed driving, low-speed driving, Bluetooth unlocking, etc.), thereby reducing the overall system response time. Furthermore, the deep learning model can continuously learn and update process priorities in real time.
[0035] In obtaining the optimal priority combination based on the process prioritization method of deep reinforcement learning, this application defines a deep reinforcement learning model, often referred to as an agent. Its goal is to learn a good policy, make optimal choices, and maximize the reward. Its three core elements are state (S), action (A), and reward (R). State refers to the representation of the agent's environment and its perception of the external environment. Action is the decision made by the agent in the given environment, and the action will change accordingly based on the decision environment. Reward is the feedback on the quality of the agent's decision. Figure 2 This refers to the Markov decision process in the DQN-PPO algorithm. Figure 2 In this context, State represents the parameter state space, Action represents the action space for transitioning between parameter state spaces, and Reward represents the immediate reward obtained by taking Action in State.
[0036] Regarding the design of the state space, it can be understood that the environment in which the deep reinforcement learning model exists includes two aspects: the driving scenario and process priority. The core of the state design in this application is the design of process priority. The Linux kernel defines a queue of runnable processes of type `struct rq` for the processor, and initializes the priority configuration [P0, P1, ..., Pn] of this queue. In the Linux system, priority is implemented by setting the `nice` value, which ranges from -20 to +19. A higher `nice` value means a lower priority, and a lower `nice` value will receive more processor time. During the learning process, the agent modifies this priority to obtain higher rewards. For example, for a... Figure 3 The five processes shown are assigned priorities [P0, P1, P2, P3, P4] = [16, 18, -18, -20, 15]. In a system with multiple processors, multiple priority configurations can be chained together, and the possible combinations of process priorities can be designed as corresponding state representations.
[0037] Regarding the design of the motion space, the motion space is represented as follows:
[0038] A = [a0, a1, ..., at, ..., ak] (Formula 1)
[0039] A is the set of priorities assigned to all processes, where each priority configuration P = [P0, P1, ..., Pn] corresponds to an action at, and k is the number of actions.
[0040] Vehicle driving scenarios include usage scenarios such as automatic parking across floors, reversing into a parking space, high-speed driving, low-speed driving, and Bluetooth unlocking.
[0041] First, this application adds the current state St, the reward value Rt corresponding to the current state, the action At corresponding to the current state, and the next state S(t+1) as an experience sequence to the experience replay pool D. That is, the experience sequence [St, At, Rt, S(t+1)] is stored in the experience replay pool D. Among them, the current state of multiple process priority combinations is St, the reward value of the current process priority combination is Rt, the action At is configured with priority, and the next state S(t+1) of the process priority combination is configured with priority based on the action.
[0042] Understandably, when configuring process priorities, it is first necessary to initialize the value table Q(s, a) corresponding to state s obtained from the value network output, and randomly select a process priority combination P with initialization parameters. Based on this initial process priority combination P, the initial state S0, the reward value R0 corresponding to the initial state, the action A0 executed in the initial state, and the next state S1 corresponding to the initial state S0 obtained after executing action A0 are determined. Then, the initial state S0, the reward value R0, the action A0, and the next state S1 are added to the experience replay pool D as an experience sequence. Using state S1 as the current state, the reward value R1 corresponding to state S1, the action A1 executed in state S1, and the next state S2 corresponding to state S1 obtained after executing action A0 are obtained. This state S1, the reward value R1, the action A1, and the next state S2 are added to the experience replay pool D as an experience sequence. This process is repeated multiple times to obtain multiple experience sequences, i.e., multiple current states S0 of multiple process priority combinations running in the vehicle driving scenario are obtained. t The reward value R of the current process priority combination t Action A for configuring priority t And according to action A t The next state S of the process priority combination after configuring priorities t+1It is added to the experience replay pool D as an experience sequence.
[0043] Then, a preset number of experience sequences are selected from the experience replay pool and processed to obtain the target priority combination. For example, if the preset number is m, then m samples are randomly selected from the experience replay pool D and processed to obtain the optimal parameter configuration combination Pbest, which is the target priority combination Pbest.
[0044] Finally, based on the target priority, Pbest manages multiple processes running in the corresponding vehicle driving scenario.
[0045] Thus, this application utilizes the computing power of the XPU to establish an experience replay pool by combining the state of process priority combinations, the corresponding reward values, and the actions of configuring process priorities. It then processes the data in the experience replay pool to find the optimal priority configuration, i.e., the target priority combination, and manages multiple processes running in the corresponding vehicle driving scenario based on this target priority combination. This eliminates the need for manual configuration and makes the priority configuration of multiple processes more accurate.
[0046] For more details, please refer to Figure 4 Step 01 includes:
[0047] 011: Obtain the raw reward value for each current state;
[0048] 012: Output the Q-value for each current state through the value network;
[0049] 013: If the next state is not the final state, determine the reward value based on the original reward value and the Q value.
[0050] The processor is used to obtain the raw reward value for each current state; output the Q value for each current state through the value network; and determine the reward value based on the raw reward value and the Q value if the next state is not the final state.
[0051] Specifically, we can first obtain the original reward value R for each current state. j Then, the Q-value of each current state is output through the value network. Then, in the next state S... j+1 In cases where it is not the final state, based on the original reward value R j The reward value y is determined by the Q value. j .
[0052] The network parameters can be obtained by training a deep reinforcement learning model's value network. This value network can be a deep neural network, where the Q-value of each current state corresponds to the Q-value of each state in the value network. The priority of each state is determined based on the state with the highest Q-value. Here, Q is a probability value, and the state S with the highest Q-value is the state the user is most likely to choose.
[0053] Further, please refer to Figure 5 Step 011 includes:
[0054] 0111: Get the total response time of multiple processes corresponding to each current state;
[0055] 0112: If the total response time exceeds the preset time threshold, the original reward value is determined to be the first set value; or
[0056] 0113: If the total response time is not greater than the preset time threshold, then the original reward value is determined to be the second set value.
[0057] The processor is used to obtain the total response time of multiple processes corresponding to each current state; if the total response time is greater than a preset time threshold, the original reward value is determined to be a first set value; or if the total response time is not greater than the preset time threshold, the original reward value is determined to be a second set value.
[0058] Specifically, this application defines the response time of each process as t. i (Ignoring the time cost of process context switching), the total response time of multiple processes corresponding to each current state can be calculated using Formula 2:
[0059]
[0060] Where n represents the number of processes, T process t represents the total response time. i This indicates the response time for each process.
[0061] The goal of a deep learning model is to minimize t, thereby maximizing the reward. Since each process requires a different execution time, assigning a priority to each process allows for the immediate calculation of the overall response time T. process .
[0062] If the total response time exceeds a preset time threshold, the original reward value is determined to be the first set value; or if the total response time is not greater than the preset time threshold, the original reward value is determined to be the second set value. For example, the preset time threshold is t. lim If the first reward setting is -10 and the second reward setting is 1, then when T process >t lim When T is 10, the reward is -10. process <t lim When the reward is 1, the reward is 1.
[0063] More specifically, t can be lim Initialize to 100ms. After an action is executed, calculate the total response time T of multiple processes corresponding to each current state.process Return the original reward value R j The reward setting.
[0064] The reward value can be set according to Formula 3:
[0065] R = [r0, r1, ..., r t ,...,r k ] (Formula 3)
[0066] Where R represents the set of rewards obtained after one round of priority configuration, k is the number of rewards, and r t To execute a t The subsequent evaluation and reward.
[0067] Thus, this application can select an action a according to the ∈-greedy policy, execute action a, first obtain the total response time Tprocess of multiple processes corresponding to each current state of the system, then obtain the immediate reward Rt according to the above reward mechanism, and then observe the new state S(t+1).
[0068] Further, please refer to Figure 6 Step 013 includes:
[0069] 0131: The reward value is obtained by adding the discount factor to the original reward value and the product of the maximum Q value that has been output.
[0070] The processor is used to add the discount factor to the original reward value and the product of the maximum Q value that has been output to obtain the reward value.
[0071] In other words, this application achieves optimized learning of the state-action value function based on the following iterative formula 4:
[0072]
[0073] In the above formula, y j R is the reward value. j The original reward value for each current state, γ is the discount factor, and max a Q(S j+1 ,a j+1 ;θ) represents the parameters of the deep reinforcement learning model, namely θ and the next state S. j+1 It is a non-final state and the next action is a. j+1 The maximum Q value has been output under the given condition. Here, γ can be 0.9.
[0074] When the next state is a non-final state, the original reward value R can be calculated according to Formula 4. j The reward value y for each action is obtained by multiplying the discount factor γ by the maximum Q value already output. jThen, the optimal reward value is selected as the decision.
[0075] Please see Figure 7 Step 01 includes:
[0076] 014: A greedy strategy is used to obtain the action with each configuration priority, where the greedy value of the greedy strategy is inversely proportional to the number of times it is obtained.
[0077] Understandably, because under normal circumstances, an agent can only choose one action at a time, if it always chooses the action with the highest reward from the previous iteration, it is easy to get trapped in a local optimum and may even fail to converge. Therefore, this application adopts a greedy strategy, where the greedy value ε is inversely proportional to the number of times j is selected, i.e., setting... This causes ε to gradually decrease with the number of attempts, preventing the model from getting trapped in local optima. Furthermore, the ∈-greedy strategy can maximize the expected future reward, giving the model a probability of exploring new actions at each selection.
[0078] In other words, this application can select the next action according to Formula 5:
[0079]
[0080] In the above formula, A t Let ε be the probability of the next action, and a0, a1, ..., a t ,...,a k There are k actions, and 'a' is an arbitrarily chosen action.
[0081] That is, the deep learning model in this application randomly selects one action from k actions with probability ε as the decision, and takes batch = 100 actions from the experience replay pool with probability 1-ε. Then, as shown in Formula 4, it calculates the action based on the original reward value R. j The Q-value determines the non-final state reward value y. j Then, the optimal reward value is selected as the decision.
[0082] The following details how to process the experience sequences selected from the experience replay pool to obtain the target priority combination Pbest.
[0083] Please see Figure 8 Step 02 includes:
[0084] 021: Select a predetermined number of experience sequences from the experience replay pool to train the deep reinforcement learning model;
[0085] 022: Calculate the loss value of the deep reinforcement learning model;
[0086] 023: Update the parameters of the deep reinforcement learning model based on the loss value;
[0087] 024: Determine the target priority combination based on the updated deep reinforcement learning model.
[0088] The processor is used to select a preset number of experience sequences from the experience replay pool to train the deep reinforcement learning model; calculate the loss value of the deep reinforcement learning model; update the parameters of the deep reinforcement learning model based on the loss value; and determine the target priority combination based on the updated deep reinforcement learning model.
[0089] It should be noted that deep reinforcement learning models can include the aforementioned value network used to output Q-values.
[0090] The preset number can be the value set by default for deep reinforcement learning models.
[0091] First, a preset number of experience sequences are selected from the experience replay pool to train the deep reinforcement learning model. The selected preset number of experience sequences can be trained according to Formulas 4 and 5 above.
[0092] Then, this application can calculate the loss value L(θ) of the deep reinforcement learning model and update the parameters of the deep reinforcement learning model based on the loss value L(θ).
[0093] Finally, the target priority combination Pbest is determined based on the updated deep reinforcement learning model.
[0094] Thus, the deep reinforcement learning model of this application can continue to learn and realize real-time updates of process priorities, that is, it can realize real-time updates of process priorities according to different driving scenarios.
[0095] For more details, please refer to Figure 9 Step 022 includes:
[0096] 0221: Output the Q-value for each current state using a deep reinforcement learning model;
[0097] 0222: Calculate the loss value of the deep reinforcement learning model based on the reward value and the corresponding Q value.
[0098] The processor is used to output the Q-value of each current state through the deep reinforcement learning model; and to calculate the loss value of the deep reinforcement learning model based on the reward value and the corresponding Q-value.
[0099] In other words, this application can first output the Q-value of each current state through a deep reinforcement learning model, thus obtaining Q(s). j ,a j ;θ), where j is the number of current states.
[0100] Then, this application can calculate the loss value L(θ) according to the gradient descent method:
[0101] L(θ)=(y j -Q(s j ,a j ;θ)) 2 (Formula 6)
[0102] Where L(θ) is the loss value, θ is the parameter of the deep reinforcement learning model, and y j For the reward value, Q(s) j ,a j ;θ) is the Q value of each current state, and j is the number of current states.
[0103] Furthermore, the parameters of the deep reinforcement learning model can be updated based on the loss value L(θ) using a formula.
[0104] Equation 7 is implemented as follows:
[0105]
[0106] Where θ represents the parameters of the deep reinforcement learning model, Q(s) j ,a j ;θ) is the Q value of the current state, y j -Q(s j ,a j ;θ) is the square root of L(θ), and β is the learning rate. The learning rate in this application can be β = 0.01.
[0107] The following section explains how to determine whether a deep reinforcement learning model has been updated.
[0108] Specifically, please refer to Figure 9 Step 024 includes:
[0109] 0241: If the number of updates to the deep reinforcement learning model reaches the preset number, then the deep reinforcement learning model update is considered complete.
[0110] 0242: Determine the target priority combination based on the updated deep reinforcement learning model.
[0111] The processor determines that the deep reinforcement learning model update is complete if the number of updates reaches a preset number; and determines the target priority combination based on the updated deep reinforcement learning model.
[0112] It is understandable that updating the parameters of a deep reinforcement learning model each time involves selecting a predetermined number of experience sequences from the experience replay pool. Since the sampled data is limited, updating the deep reinforcement learning model only once may not yield the optimal parameter combination. Therefore, this application achieves iterative updates to the deep reinforcement learning model. That is, after each update, the value network of the updated deep reinforcement learning model is used to repeatedly acquire the corresponding experience sequences and add them to a new experience replay pool. The new experience replay pool is then used to retrain the deep reinforcement learning model and update it, thus achieving iterative updates of the deep reinforcement learning model.
[0113] In each iteration of the deep reinforcement learning model, the reward value corresponding to the priority combination is Rnow = Rbefore + R(S, A), where Rbefore is the sum of reward values obtained from previous priority iterations, and R(S, A) is the reward value corresponding to the current priority combination. Furthermore, if Rnow > Rmax, then R_max = R_now, where Rmax is the maximum reward value among the completed priority iterations. In other words, if the reward value corresponding to the priority combination after the deep reinforcement learning model update is greater than the maximum reward value among the completed priority iterations, then the maximum reward value is used for the update.
[0114] This application can determine that the deep reinforcement learning model update is complete when the number of updates reaches a preset number, and can determine the target priority combination based on the updated deep reinforcement learning model.
[0115] This application also provides a non-volatile computer-readable storage medium containing a computer program. When the computer program is executed by one or more processors, it implements the vehicle process management method described in any of the above embodiments.
[0116] For example, when a computer program is executed by a processor, it implements the following steps of a vehicle process management method:
[0117] 01: The current state of multiple process priority combinations running in the vehicle driving scenario, the reward value of the current process priority combination, the action of configuring the priority, and the next state of the process priority combination after configuring the priority according to the action are obtained multiple times and added to the experience replay pool as an experience sequence.
[0118] 02: Select a preset number of experience sequences from the experience replay pool and process them to obtain the target priority combination;
[0119] 03: Manage multiple processes running in the corresponding vehicle driving scenario based on target priority.
[0120] It is understood that a computer program includes computer program code. Computer program code can be in the form of source code, object code, executable files, or some intermediate form. Computer-readable storage media can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), and software distribution media, etc.
[0121] The storage medium of this application utilizes the vehicle process management method described above to train a deep reinforcement learning model using the powerful computing power of the XPU system. It performs priority configuration combination training to find the optimal priority configuration, i.e., the target priority combination, and manages multiple processes running in the corresponding vehicle driving scenario according to the target priority combination. This eliminates the need for manual configuration and makes the priority configuration of multiple processes more accurate.
Claims
1. A vehicle process management method, characterized in that, include: The current state of multiple process priority combinations running in the vehicle driving scenario, the reward value of the current process priority combination, the action of configuring the priority, and the next state of the process priority combination after configuring the priority according to the action are obtained multiple times and added to the experience replay pool as an experience sequence. A predetermined number of experience sequences are selected from the experience replay pool and processed to obtain a target priority combination; Multiple processes running in the corresponding vehicle driving scenario are managed according to the target priority combination; The step of selecting a preset number of experience sequences from the experience replay pool and processing them to obtain a target process priority combination includes: A predetermined number of experience sequences are selected from the experience replay pool to train the deep reinforcement learning model; The deep reinforcement learning model outputs the Q-value for each current state, where the Q-value is the probability value that the user selects each current state. The loss value of the deep reinforcement learning model is calculated based on the reward value and the corresponding Q value; The parameters of the deep reinforcement learning model are updated based on the loss value; The target priority combination is determined based on the updated deep reinforcement learning model.
2. The vehicle process management method according to claim 1, characterized in that, The process of repeatedly acquiring the current state of multiple process priority combinations running in a vehicle driving scenario, the reward value of the current process priority combination, the action of configuring the priority, and the next state of the process priority combination after configuring the priority according to the action, as an experience sequence added to the experience replay pool, includes: Obtain the original reward value for each of the current states; The Q-value of each current state is output through a value network, and the Q-value is the probability value of each current state being selected by the user. If the next state is not the final state, the reward value of the next state is determined based on the original reward value and the Q value.
3. The vehicle process management method according to claim 2, characterized in that, The step of obtaining the original reward value for each current state includes: Obtain the total response time of the multiple processes corresponding to each current state; If the total response time is greater than a preset time threshold, then the original reward value is determined to be a first set value; or If the total response time is not greater than a preset time threshold, then the original reward value is determined to be the second set value.
4. The vehicle process management method according to claim 2, characterized in that, When the next state is not the final state, determining the reward value based on the original reward value and the Q value includes: The reward value is obtained by adding the discount factor to the original reward value and the product of the maximum Q value that has been output, where the maximum Q value is the largest among the Q values of each of the current states that have been output.
5. The vehicle process management method according to claim 1, characterized in that, The process of repeatedly acquiring the current state of multiple process priority combinations running in a vehicle driving scenario, the reward value of the current process priority combination, the action of configuring the priority, and the next state of the process priority combination after configuring the priority according to the action, as an experience sequence added to the experience replay pool, includes: A greedy strategy is used to obtain the action with each configured priority, wherein the greedy value of the greedy strategy is inversely proportional to the number of times it is obtained.
6. The vehicle process management method according to claim 1, characterized in that, Determining the target priority combination based on the updated deep reinforcement learning model includes: If the deep reinforcement learning model is updated a preset number of times, then the deep reinforcement learning model update is determined to be complete. The target priority combination is determined based on the updated deep reinforcement learning model.
7. A vehicle, characterized in that, The vehicle includes a processor and a memory, the memory storing a computer program that, when executed by the processor, implements the vehicle process management method according to any one of claims 1-6.
8. A non-volatile computer-readable storage medium containing a computer program, characterized in that, When the computer program is executed by one or more processors, it implements the vehicle process management method according to any one of claims 1-6.