An adaptive cruise decision-making method fusing Q algorithm and fuzzy state division

Abstract

The application provides a self-adaptive cruise decision method combining Q algorithm and fuzzy state division, comprising the following steps: step 1, establishing a discrete state variable model; step 2, establishing a fuzzy variable model; step 3, initializing the value in the Q matrix, setting the learning rate alpha, the attenuation rate gamma and the reward value r; step 4, calculating the fitness mu i (s) and selecting the state with the maximum fitness mu k (s); step 5, selecting the action to be executed b; step 6, executing the action b to obtain the next time state s' and the reward value r; step 7, calculating the confidence degree omega j and updating the Q value; step 8, discretizing the continuous action; step 9, judging whether the stopping condition of the self-adaptive cruise task is met, if yes, the self-adaptive cruise task is stopped, if not, the step of calculating the fitness is entered again. The method solves the problems that the traditional algorithm has poor effect due to insufficient training samples and unbalanced collection and cannot be implemented due to the limited storage space of the matrix.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned vehicle control, and particularly relates to an adaptive cruise decision-making method fusing Q algorithm and fuzzy state division. BACKGROUND

[0002] In recent years, artificial intelligence systems have been increasingly concerned and widely applied. Artificial intelligence is a new technology that integrates multiple scientific disciplines and interacts with a large number of technical fields. At present, with the rapid development of chip technology, computer big data technology and battery technology, artificial intelligence technology has also been rapidly improved, and products and concepts related to it have gradually been integrated into various fields of society. The concept of artificial intelligence has inevitably been introduced into the field of automatic driving. The intelligent adaptive cruise system in the field of automatic driving not only reduces the complexity of the driver's operation, but also more efficiently and accurately completes the driving task, which requires strong artificial intelligence decision-making capability.

[0003] Many scholars have carried out a lot of research work on adaptive cruise technology. For example, Wang Zeba based on the variable vehicle headway strategy of the relative speed between two vehicles, analyzed the stability of the PID control algorithm, and designed an incremental PID control algorithm accordingly. Liu Daoxudong also considered the multi-element information processing ability and strong learning ability of the human brain under the framework of the PID control algorithm, integrated it with the artificial neural network, and improved the sensitivity and efficiency of the vehicle upper controller, which has shown good results in many scenarios. Chen Yefeng and Chen Linlin et al. studied the PID control strategy, optimized the performance of the vehicle's branch stream click mechanical characteristics, and used a single-chip microcomputer to generate PWM to control the vehicle motor, which was proved to have good control effect through simulation experiments.

[0004] However, the above algorithms lack automatic reasoning and generalization ability due to the lack of artificial intelligence technology, and the training effect of traditional machine learning algorithms is strongly dependent on training samples, which often leads to poor results due to insufficient number of training samples and unbalanced collection.

[0005] In addition, in the traditional Q-learning algorithm, the state-action value is stored in the form of a matrix, which is not feasible in many cases in actual scenarios because the state quantity in complex scenarios is too large and the storage space of the matrix is limited. SUMMARY

[0006] In order to solve the problems that the training effect of the traditional algorithm depends on the training samples, which often leads to poor results due to insufficient number of training samples and unbalanced collection, and the traditional algorithm cannot be implemented because the state quantity in complex scenarios is too large and the storage space of the matrix is limited, the purpose of the present application is to provide an adaptive cruise decision-making method fusing Q-learning and fuzzy state division.

[0007] The method can train the "brain" of the agent, the vehicle can try and error in various scenes, automatically generate a large amount of effective data, save a large amount of time cost and labor cost, achieve the effect of greatly expanding the neural network training sample, so as to obtain sufficient experience knowledge, learn which decision is good and which decision is bad through the reward value obtained in different states, the algorithm supplements a large amount of data for the traditional machine learning algorithm, and makes up for the short board.

[0008] In addition, the present application fuses the fuzzy mathematics algorithm to compress the dimensions of the state and action space, greatly saves the space complexity required by the algorithm, and improves the training speed.

[0009] To achieve the above object, the technical scheme of the present application is as follows.

[0010] A self-adaptive cruise decision method fusing Q algorithm and fuzzy state division, comprising the following steps:

[0011] Step 1, establishing a discretized state variable model;

[0012] Step 2, establishing a fuzzy variable model of state and action;

[0013] Step 3, initializing the value in the Q matrix according to the fuzzy variable model established in step 2, and setting the learning rate α, the decay rate γ and the reward value r;

[0014] Wherein, Wherein, a is acceleration, and s is a two-dimensional state variable;

[0015] Step 4, calculating the fitness μ i (s) and selecting the state with the maximum fitness μ k (s);

[0016] Wherein,

[0017] Step 5, selecting the action b to be executed from the kth row in the Q matrix in step 3;

[0018] Step 6, executing the action b to obtain the next time state s' and the reward value r;

[0019] Step 7, calculating the confidence ω j and updating the Q value;

[0020] Wherein,

[0021] Q(s k ,a l )=Q(s k ,al )+ a(r + gωQ'(s k′ , a l′ )), s k ← s k′ ; where a is acceleration, s is two-dimensional state variable;

[0022] Step 8, discretization of the action is continuous to obtain the adaptive cruise task completion;

[0023]

[0024] Step 9, according to the adaptive cruise task completion of step 8, whether to meet the adaptive cruise task stop condition, if meet the adaptive cruise task is stopped, if not meet again into step 4 of the calculation of fitness step.

[0025] Further, in step 1, the discretization of the state variable model is as follows:

[0026]

[0027] Where v pre is the first vehicle speed, is the first vehicle speed of the discrete state variable;

[0028]

[0029] Where v host is the second vehicle speed, is the second vehicle speed of the discrete state variable;

[0030]

[0031] Where x real is the actual distance between the two vehicles, is the actual distance between the two vehicles of the discrete state variable;

[0032]

[0033] Where S 3 is combined into a three-dimensional state variable, S 1 is a one-dimensional state variable after mapping S 3 .

[0034] Further, in step 2, before establishing the fuzzy variable model of state and action, the three-dimensional state variable S 3 is converted into a two-dimensional state variable s * :

[0035] s * = (x ev e )

[0036] x e =(x real -x safe ) / x safe

[0037] v e =v host -v pre ;

[0038] wherein x real is the actual distance between the two vehicles; x safe is the safe distance between the two vehicles; v host is the speed of the second vehicle, and v pre is the speed of the first vehicle.

[0039] Further, in step 2, the fuzzy variable model includes a fuzzy variable model of distance relative error x e , a fuzzy variable model of speed error v e , and a fuzzy variable model of acceleration a.

[0040] Further, for the fuzzy variable model of distance relative error x e and the fuzzy variable model of speed error v e , seven fuzzy state variables are set, and the seven fuzzy state variables are respectively negative large, negative medium, negative small, zero, positive small, positive medium, and positive large.

[0041] The fuzzy variable model of distance relative error x e is as follows:

[0042] (1) When x e is lower than -0.28, the membership degree of x e for negative large is 1; when x e is greater than -0.16, the membership degree of x e for negative large is 0; accordingly, the triangular function value of negative large is set as [-∞, -0.28, -0.16];

[0043] (2) When x e is lower than -0.28 or higher than -0.08, the membership degree of x e for negative medium is 0; when x e is -0.16, the membership degree of x e for negative medium is 1; accordingly, the triangular function value of negative medium is set as [-0.28, -0.16, -0.08];

[0044] (3) When x e is lower than -0.16 or higher than 0, the membership degree of x eFor negative small membership degrees, the membership degree is 0; when x e When x is -0.08, e The membership degree for negative small values ​​is 1; therefore, the trigonometric function value for negative small values ​​is set to [-0.16, -0.08, 0].

[0045] (4) When x e When x is below -0.08 or above 0.08, the membership degree of zero is 0; when x e When the value is 0, the membership degree of zero is 1; therefore, the trigonometric function value of zero is set to [-0.08, 0, 0.08].

[0046] (5) When x e When x is below 0 or above 0.16, e For positive integers, the membership degree is 0; when x is small, the membership degree is 0. e When x is 0.08, e The membership degree of positive smallest is 1; therefore, the trigonometric function value of positive smallest is set to [0, 0.08, 0.16].

[0047] (6) When x e When x is below 0.08 or above 0.28, e The membership degree is 0 for the middle; when x e When x is 0.16, e The membership degree of the center is 1; therefore, the trigonometric function value of the center is set to [0.08, 0.16, 0.28].

[0048] (7) When x e When x is below 0.16, e The membership degree of Zhengda is 0; x e When x is greater than 0.28, e The membership degree of Zhengda is 1; therefore, the trigonometric function value of Zhengda is set to [0.16, 0.28, ∞].

[0049] Speed ​​error v e The fuzzy variable model is as follows:

[0050] (1) When v e When below -4, v e For negative values, the membership degree is 1; when v e When it is greater than -2, v e The membership degree of negative large values ​​is 0; therefore, the trigonometric function value of negative large values ​​is set to [-∞, -4, -2].

[0051] (2) When v e When below -4 or above -1, v e The membership degree is 0 for negative values; when v e When v is -2, eFor negative membership of 1; accordingly, the triangular function value of negative is set as [-4, -2, -1];

[0052] (3) When v e is lower than -2 or higher than 0, v e For negative small membership of 0, when v e is exactly -1, v e For negative small membership of 1; accordingly, the triangular function value of negative small is set as [-2, -1, 0];

[0053] (4) When v e is lower than -1 or higher than 1, v e For zero membership of 0; when v e is 0, v e For zero membership of 1; accordingly, the triangular function value of zero is set as [-1, 0, 1];

[0054] (5) When v e is lower than 0 or higher than 2, v e For positive small membership of 0; when v e is 1, v e For positive small membership of 1; accordingly, the triangular function value of positive small is set as [0, 1, 2];

[0055] (6) When v e is lower than 1 or higher than 4, v e For positive medium membership of 0; when v e is 2, v e For positive medium membership of 1; accordingly, the triangular function value of positive medium is set as [1, 2, 4];

[0056] (7) When v e is lower than 2, v e For positive large membership of 0; when v e is greater than 4, v e For positive large membership of 1; accordingly, the triangular function value of positive large is set as [2, 4, ∞].

[0057] Further, for the fuzzy variable model of acceleration a, nine fuzzy state variables are set, and the nine fuzzy state variables are respectively negative very large, negative large, negative medium, negative small, zero, positive small, positive medium, positive large, and positive very large;

[0058] The fuzzy variable model of acceleration a is as follows:

[0059] (1) When a is lower than -2.5 or higher than 0.7, a is 0 for the membership of very negative; when a is greater than -2, a is 1 for the membership of very negative; accordingly, the triangular function value of very negative is set as [-2.5, -2, -0.7];

[0060] (2) When a is lower than -1.2 or higher than -0.3, a is 0 for the membership of negative; when a is equal to -0.7, a is 1 for the membership of negative; accordingly, the triangular function value of negative is set as [-1.2, -0.7, -0.3];

[0061] (3) When a is lower than -0.7 or higher than -0.1, a is 0 for the membership of negative middle; when a is -0.3, a is 1 for the membership of negative middle; accordingly, the triangular function value of negative middle is set as [-0.7, -0.3, -0.1];

[0062] (4) When a is lower than -0.3 or higher than 0, a is 0 for the membership of negative small; when a is -0.1, a is 1 for the membership of negative small; accordingly, the triangular function value of negative small is set as [-0.3, -0.1, 0];

[0063] (5) When a is lower than -0.1 or higher than 0.1, a is 0 for the membership of zero; when a is 0, a is 1 for the membership of zero; accordingly, the triangular function value of zero is set as [-0.1, 0, 0.1];

[0064] (6) When a is lower than 0 or higher than 0.2, a is 0 for the membership of positive small; when a is 0.1, a is 1 for the membership of positive small; accordingly, the triangular function value of positive small is set as [0, 0.1, 0.2];

[0065] (7) When a is lower than 0.1 or higher than 0.5, a is 0 for the membership of positive middle; when a is 0.2, a is 1 for the membership of positive middle; accordingly, the triangular function of positive middle is set as [0.1, 0.2, 0.5];

[0066] (8) When a is lower than 0.2 or higher than 0.9, a is 0 for the membership of positive large; when a is 0.5, a is 1 for the membership of positive large; accordingly, the triangular function of positive large is set as [0.2, 0.5, 0.9];

[0067] (9) When a is lower than 0.5 or higher than 1.5, a is 0 for the membership of positive very large; when a is 0.9, a is 1 for the membership of positive very large; accordingly, the triangular function of positive very large is set as [0.5, 0.9, 1.5].

[0068] Further, in step 6, after the action b is performed, the discretized state variable model is updated as follows:

[0069]

[0070] wherein, is the action of the second vehicle; and are the discrete state variables of the second vehicle speed at different times, respectively; is the action of the first vehicle; and are the discrete state variables of the second vehicle speed at different times, respectively; is the discrete state variable of the actual distance between the two vehicles; τ = 1.

[0071] Further, in step 6, after the action b is executed, the reward value r obtained is calculated according to the following formula:

[0072] r = r g + r p ;

[0073] wherein, r g is a target reward value, r g rewarding or punishing whether the target is reached in the adaptive cruise task; r g is set as follows:

[0074]

[0075] r p is a process reward value, r p rewarding or punishing whether a better state is tended to in the adaptive cruise process; r p is set as follows:

[0076]

[0077] wherein, x real is the actual distance between the two vehicles; x safe is the safe distance between the two vehicles.

[0078] Further, in step 6, after the action b is executed, the next time state s' obtained is set as follows:

[0079]

[0080] wherein, v host is the second vehicle speed, and x safe is the safe distance between the two vehicles.

[0081] Advantages of the present application:

[0082] 1. The method of the present application first proposes a new state space division method for adaptive cruise control system, and reasonably divides the state space of the adaptive cruise control system through fuzzy mathematics algorithm, so that the Q-learning algorithm can be effectively applied to the adaptive cruise control system of the unmanned vehicle. Then, the heuristic reward value function is reasonably set based on various factors between two vehicles, and the training effect of the Q-learning algorithm is enhanced. Finally, through the computer simulation environment and the intelligent vehicle simulation experiment platform, various adaptive cruise scenes are set, and the above-mentioned algorithm of the present application is verified respectively. According to the test effect, it is shown that the method can well complete the adaptive cruise task, and has good applicability and robustness. The method of the present application solves the problems that the training effect of the traditional algorithm depends on the training sample, and the effect is poor due to insufficient and unbalanced collection of the training sample, and the problem that the traditional algorithm cannot be implemented due to too large state quantity in complex scenes and limited matrix storage space.

[0083] 2. The method of the present application can train the "brain" of the intelligent agent, and the vehicle can try and error in various scenes, automatically generate a large amount of effective data, save a large amount of time cost and labor cost, achieve a strong effect of expanding the neural network training sample, so as to obtain sufficient experience knowledge, learn which decision is good and which decision is bad through the reward value obtained in different states, and the method supplements a large amount of data for the traditional machine learning algorithm and makes up for the short board.

[0084] 3. The method of the present application fuses the fuzzy mathematics algorithm to compress the dimensions of the state and action space, greatly saves the space complexity required by the method, and improves the training speed. BRIEF DESCRIPTION OF DRAWINGS

[0085] Figure 1 The flow chart of the adaptive cruise decision method fusing the Q algorithm and the fuzzification state division provided for the embodiment 1 of the present application.

[0086] Figure 2 The speed tracking curve of the simulation test for the embodiment 2 of the present application.

[0087] Figure 3 The distance tracking curve of the simulation test for the embodiment 2 of the present application. DETAILED DESCRIPTION

[0088] In order to make the purpose, technical scheme and advantages of the present application more clear and understandable, the present application is further described in detail below in combination with embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0089] All other embodiments obtained by a person of ordinary skill in the art without creative labor based on the embodiments of the present application fall within the scope of protection of the present application.

[0090] The experimental methods in the following embodiments are routine methods unless otherwise specified; the reagents and materials can be purchased on the market unless otherwise specified.

[0091] Embodiment 1

[0092] Please refer to Figure 1 An adaptive cruise decision-making method combining Q algorithm and fuzzy state division, comprising the following steps:

[0093] Step 1, establishing a discretized state variable model;

[0094] Specifically, the discretized state variable model is as follows:

[0095]

[0096] wherein, v pre is the speed of the first vehicle, is the discretized state variable of the speed of the first vehicle;

[0097]

[0098] wherein, v host is the speed of the second vehicle, is the discretized state variable of the speed of the second vehicle;

[0099]

[0100] wherein, x real is the actual distance between the two vehicles, is the discretized state variable of the actual distance between the two vehicles;

[0101]

[0102] wherein, S 3 is a three-dimensional state variable composed of S 1 is a one-dimensional state variable obtained by mapping S 3 .

[0103] Step 2, first convert the three-dimensional state variable S 3 into a two-dimensional state variable s * :

[0104] s * = (x e , v e )

[0105] x e = (x real - x safe ) / x safe

[0106] v e = v host - v pre ;

[0107] wherein, x real is the actual distance between two vehicles; x safe is the safe distance between two vehicles; v host is the speed of the second vehicle, and v pre is the speed of the first vehicle.

[0108] Then, the fuzzy variable models of the states and actions are established; the fuzzy variable models include the fuzzy variable model of the distance relative error x e , the fuzzy variable model of the speed error v e , and the fuzzy variable model of the acceleration a.

[0109] For the fuzzy variable model of the distance relative error x e and the fuzzy variable model of the speed error v e , seven fuzzy state variables are set, and the seven fuzzy state variables are negative large, negative medium, negative small, zero, positive small, positive medium, and positive large, respectively.

[0110] The fuzzy variable model of the distance relative error x e is as follows:

[0111] (1) When x e is lower than -0.28, the membership degree of x e for the negative large is 1; when x e is greater than -0.16, the membership degree of x e for the negative large is 0; accordingly, the triangular function value of the negative large is set as [-∞, -0.28, -0.16];

[0112] (2) When x e is lower than -0.28 or higher than -0.08, the membership degree of x e for the negative medium is 0; when x e is -0.16, the membership degree of x e for the negative medium is 1; accordingly, the triangular function value of the negative medium is set as [-0.28, -0.16, -0.08];

[0113] (3) When x e is lower than -0.16 or higher than 0, the membership degree of x e for the negative small is 0; when x e is -0.08, the membership degree of x eThe membership degree for negative small values ​​is 1; therefore, the trigonometric function value for negative small values ​​is set to [-0.16, -0.08, 0].

[0114] (4) When x e When x is below -0.08 or above 0.08, the membership degree of zero is 0; when x e When the membership value is 0, the membership degree of zero is 1; therefore, the trigonometric function value of zero is set to [-0.08, 0, 0.08].

[0115] (5) When x e When x is below 0 or above 0.16, e For positive integers, the membership degree is 0; when x is small, the membership degree is 0. e When x is 0.08, e The membership degree of positive smallest is 1; therefore, the trigonometric function value of positive smallest is set to [0, 0.08, 0.16].

[0116] (6) When x e When x is below 0.08 or above 0.28, e The membership degree is 0 for the middle; when x e When x is 0.16, e The membership degree of the center is 1; therefore, the trigonometric function value of the center is set to [0.08, 0.16, 0.28].

[0117] (7) When x e When x is below 0.16, e The membership degree of Zhengda is 0; x e When x is greater than 0.28, e The membership degree of Zhengda is 1; therefore, the trigonometric function value of Zhengda is set to [0.16, 0.28, ∞].

[0118] Speed ​​error v e The fuzzy variable model is as follows:

[0119] (1) When v e When below -4, v e For negative values, the membership degree is 1; when v e When it is greater than -2, v e The membership degree of negative large values ​​is 0; therefore, the trigonometric function value of negative large values ​​is set to [-∞, -4, -2].

[0120] (2) When v e When below -4 or above -1, v e The membership degree is 0 for negative values; when v e When v is -2, e The membership degree of negative values ​​is 1; therefore, the trigonometric function values ​​of negative values ​​are set to [-4, -2, -1].

[0121] (3) When v e is below -2 or above 0, v e is 0 for negative small membership; when v e is exactly -1, v e is 1 for negative small membership; accordingly, the negative small triangular function value is set as [-2, -1, 0];

[0122] (4) When v e is below -1 or above 1, v e is 0 for zero membership; when v e is 0, v e is 1 for zero membership; accordingly, the zero triangular function value is set as [-1, 0, 1];

[0123] (5) When v e is below 0 or above 2, v e is 0 for positive small membership; when v e is 1, v e is 1 for positive small membership; accordingly, the positive small triangular function value is set as [0, 1, 2];

[0124] (6) When v e is below 1 or above 4, v e is 0 for positive middle membership; when v e is 2, v e is 1 for positive middle membership; accordingly, the positive middle interval is set as [1, 2, 4];

[0125] (7) When v e is below 2, v e is 0 for positive large membership; when v e is greater than 4, v e is 1 for positive large membership; accordingly, the positive large triangular function value is set as [2, 4, ∞].

[0126] For the fuzzy variable model of acceleration a, nine fuzzy state variables are set, and the nine fuzzy state variables are negative very large, negative large, negative middle, negative small, zero, positive small, positive middle, positive large, and positive very large;

[0127] The fuzzy variable model of acceleration a is as follows:

[0128] (1) When a is below -2.5 or above 0.7, a is 0 for negative very large membership; when a is greater than -2, a is 1 for negative very large membership; accordingly, the negative very large triangular function value is set as [-2.5, -2, -0.7];

[0129] (2) When a is lower than -1.2 or higher than -0.3, a has a membership of 0 for negative large; when a is equal to -0.7, a has a membership of 1 for negative large; accordingly, the triangular function value for negative large is set as [-1.2, -0.7, -0.3];

[0130] (3) When a is lower than -0.7 or higher than -0.1, a has a membership of 0 for negative middle; when a is -0.3, a has a membership of 1 for negative middle; accordingly, the triangular function value for negative middle is set as [-0.7, -0.3, -0.1];

[0131] (4) When a is lower than -0.3 or higher than 0, a has a membership of 0 for negative small; when a is -0.1, a has a membership of 1 for negative small; accordingly, the triangular function value for negative small is set as [-0.3, -0.1, 0];

[0132] (5) When a is lower than -0.1 or higher than 0.1, a has a membership of 0 for zero; when a is 0, a has a membership of 1 for zero; accordingly, the triangular function value for zero is set as [-0.1, 0, 0.1];

[0133] (6) When a is lower than 0 or higher than 0.2, a has a membership of 0 for positive small; when a is 0.1, a has a membership of 1 for positive small; accordingly, the triangular function value for positive small is set as [0, 0.1, 0.2];

[0134] (7) When a is lower than 0.1 or higher than 0.5, a has a membership of 0 for positive middle; when a is 0.2, a has a membership of 1 for positive middle; accordingly, the triangular function for positive middle is set as [0.1, 0.2, 0.5];

[0135] (8) When a is lower than 0.2 or higher than 0.9, a has a membership of 0 for positive large; when a is 0.5, a has a membership of 1 for positive large; accordingly, the triangular function for positive large is set as [0.2, 0.5, 0.9];

[0136] (9) When a is lower than 0.5 or higher than 1.5, a has a membership of 0 for positive very large; when a is 0.9, a has a membership of 1 for positive very large; accordingly, the triangular function for positive very large is set as [0.5, 0.9, 1.5].

[0137] Step 3, initialize the value in the Q matrix according to the fuzzy variable model established in step 2, and set the learning rate a, the decay rate g and the reward value r;

[0138] wherein, wherein, a is acceleration, and s is two-dimensional state quantity;

[0139] Step 4, calculate the fitness μ i(s) and select the action b with the maximum fitness μ k (s) and select the action b with the maximum fitness μ

[0140] wherein,

[0141] Step 5, select the action b to be performed from the k-th row in the Q matrix of step 3;

[0142] Step 6, perform the action b to obtain the next time state s' and the reward value r;

[0143] After performing the action b, the discretized state variable model is updated as follows:

[0144]

[0145] wherein, is the action of the second vehicle; and are the discretized state variables of the speed of the second vehicle at different times, respectively; is the action of the first vehicle; and are the discretized state variables of the speed of the second vehicle at different times, respectively; is the discretized state variable of the actual distance between the two vehicles; τ = 1.

[0146] After performing the action b, the reward value r obtained is calculated as follows:

[0147] r = r g + r p ;

[0148] wherein, r g is the target reward value, r g reward or punish for whether the target is reached in the adaptive cruise task; r g is set as follows:

[0149]

[0150] r p is the process reward value, r p reward or punish for whether the process tends to a better state in the adaptive cruise process; r p is set as follows:

[0151]

[0152] wherein, x real is the actual distance between the two vehicles; x safe is the safe distance between the two vehicles.

[0153] After performing the action b, the next time state s' obtained is set as follows:

[0154]

[0155] wherein v host is the second vehicle speed, x safe is the safety distance between the two vehicles.

[0156] Step 7, calculate the confidence ω j and update the Q value;

[0157] wherein,

[0158] Q(s k ,a l )=Q(s k ,a l )+α(r+γωQ′(s k′ ,a l′ )),s k ←s k′ ; wherein a is acceleration and s is a two-dimensional state variable;

[0159] Step 8, discretize the action to obtain the completion of the adaptive cruise task;

[0160]

[0161] Step 9, according to the completion of the adaptive cruise task in step 8, determine whether the stop condition of the adaptive cruise task is met, if yes, the adaptive cruise task is terminated, if not, enter the step 4 of calculating the fitness again.

[0162] Embodiment 2

[0163] According to the method of embodiment 1, the specific simulation results are shown in Figure 2 and Figure 3 .

[0164] From Figure 2 and Figure 3 , it can be seen that the whole adaptive cruise time is 250 seconds.

[0165] Figure 2 The speed tracking curve is shown in Figure 2 , wherein the dashed line represents the speed of the front vehicle and the solid line represents the speed of the second vehicle, it can be seen that the second vehicle (the second vehicle) changes the speed to the same as the first vehicle (the first vehicle) within 15 seconds.

[0166] Figure 3 The distance tracking curve is shown in Figure 3The middle dotted line represents the safe distance between the two vehicles, and the safe distance is set as 2 times of the speed of the second vehicle, and the solid line represents the actual distance between the two vehicles. It can be seen that within 20 seconds, the second vehicle reaches the safe distance from the first vehicle. The above is only a preferred embodiment of the present application, and is not used to limit the present application. Any modification, equivalent replacement and improvement within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An adaptive cruise decision method fusing Q-algorithm and fuzzified state partitioning, characterized in that, The method comprises the following steps: Step 1, establishing a discretized state variable model; Step 2, establishing a fuzzy variable model of state and action; Step 3, initializing the value in the Q matrix according to the fuzzy variable model established in step 2, and setting the learning rate α, the attenuation rate γ and the reward value r; wherein, wherein a is acceleration and s is a two-dimensional state variable. Step 4, calculate the fitness μ i (s) and select the state with the largest fitness μ k (s). wherein Step 5, selecting an action b to be executed from the kth row in the Q matrix in step 3; Step 6, executing the action b to obtain the next time state s' and the reward value r; Step 7, calculate the confidence ω j and update the Q value; wherein Q(s k ,a l ) = Q(s k ,a l ) + a(r + gQ'(s k′ ,a l′ )), s k ← s k′ ; where a is acceleration and s is a two-dimensional state variable. Step 8, continuous the discrete action to obtain the completion of the adaptive cruise task; Step 9, judging whether the stopping condition of the adaptive cruise task is met according to the completion of the adaptive cruise task in step 8, if yes, the adaptive cruise task is stopped, if not, the step 4 of calculating the fitness is entered again.

2. The adaptive cruise decision method of claim 1, wherein, In step 1, the discretized state variable model is as follows: where v pre is the speed of the first vehicle, is a discrete state quantity of the speed of the first vehicle; where v host is the speed of the second vehicle, is a discrete state quantity for the speed of the second vehicle; where x real is the actual distance between the two vehicles, is the discrete state of the actual distance between the two vehicles; where S 3 is a three-dimensional state quantity, S 1 is a one-dimensional state quantity obtained by mapping S 3 .

3. The adaptive cruise decision method of claim 2, wherein, In Step 2, the three-dimensional state variable S 3 is converted to a two-dimensional state variable s * before the fuzzy variable model of the state and action is established. s * = (x e ,v e ) x e = (x real - x safe ) / x safe v e = v host - v pre ; wherein x real is the actual distance between the two vehicles; x safe is the safe distance between the two vehicles; v host is the speed of the second vehicle, and v pre is the speed of the first vehicle.

4. The adaptive cruise decision method of claim 3, wherein, In step 2, the fuzzy variable model includes a fuzzy variable model of distance relative error x e , a fuzzy variable model of velocity error v e , and a fuzzy variable model of acceleration a.

5. The adaptive cruise decision method of claim 4, wherein, For the fuzzified variable model of distance relative error x e and the fuzzified variable model of velocity error v e , seven fuzzy state variables are set, and the seven fuzzy state variables are negative large, negative medium, negative small, zero, positive small, positive medium, and positive large, respectively. Distance relative error x e The fuzzified variable model is as follows: (1) When x e When x is below -0.28, e For negative values, the membership degree is 1; when x e When x is greater than -0.16, e The membership degree for negative values ​​is 0; therefore, the trigonometric function value for negative values ​​is set to [-∞, -0.28, -0.16]. (2) when x e is lower than -0.28 or higher than -0.08, x e For the membership in the negative, 0; when x e is -0.16, x e For the membership in the negative, 1; accordingly, the triangular function value in the negative is set as [-0.28, -0.16, -0.08]; (3) When x e When x is below -0.16 or above 0, e For negative small membership degrees, the membership degree is 0; when x e When x is -0.08, e The membership degree for negative small values ​​is 1; therefore, the trigonometric function value for negative small values ​​is set to [-0.16, -0.08, 0]. (4) When x e is lower than -0.08 or higher than 0.08, the membership degree of zero is 0; when x e is 0, the membership degree of zero is 1; and the triangular function value of zero is set as [-0.08, 0, 0.08] accordingly. (5) when x e is less than 0 or greater than 0.16, x e is set to 0; when x e is 0.08, x e is set to 1; from which the triangular function values for positive small are set to [0, 0.08, 0.16]; (6) when x e is below 0.08 or above 0.28, x e is set to 0; when x e is 0.16, x e is set to 1; from which the triangular function values for the positive center are set to [0.08, 0.16, 0.28]; (7) When x e When x is below 0.16, e The membership degree of Zhengda is 0; x e When x is greater than 0.28, e The membership degree of Zhengda is 1; therefore, the trigonometric function value of Zhengda is set to [0.16, 0.28, ∞]. Speed error v e The fuzzified variable model is as follows: (1) when v e is less than -4, v e is greater than -2, v e is greater than -2, v e is greater than -2, v e is greater than -2, v e is greater than -2, v e is greater than -2, v e is greater than -2, v e is greater than -2, v e is greater than -2, v e is greater than -2, v e is greater than -2, v e is greater than -2, v (2) when v e is lower than -4 or higher than -1, v e has a membership of 0 in negative; when v e is -2, v e has a membership of 1 in negative; the trigonometric function value in negative is set as [-4, -2, -1] accordingly; (3) when v e is less than -2 or greater than 0, v e is 0 for negative small membership; accordingly, the negative small triangular function value is set to [-2, -1, 0]; e is exactly -1, v e is 1 for negative small membership; accordingly, the negative small triangular function value is set to [-2, -1, 0]; (4) When v e When v is below -1 or above 1 e The membership degree is 0 for zero; when v e When v is 0, e The membership degree of zero is 1; therefore, the trigonometric function value of zero is set to [-1, 0, 1]. (5) When v e When v is below 0 or above 2 e For positive values, the membership degree is 0; when v e When v is 1, e The membership degree of positive smallest is 1; therefore, the trigonometric function value of positive smallest is set to [0, 1, 2]. (6) when v e is less than 1 or greater than 4, v e is set to 0; when v e is 2, v e is set to 1; from which the interval setting for the positive is set to [1, 2, 4]; (7) when v e is less than 2, v e For positive large membership is 0; when v e is greater than 4, v e For positive large membership is 1; from which the positive large triangle function value is set as [2, 4, ∞].

6. The adaptive cruise decision method of claim 5, wherein, For the fuzzy variable model of acceleration a, nine fuzzy state variables are set, and the nine fuzzy state variables are negative very large, negative large, negative medium, negative small, zero, positive small, positive medium, positive large and positive very large; The fuzzy variable model of acceleration a is as follows: (1) When a is lower than -2.5 or higher than 0.7, the membership degree of a to negative very large is 0; when a is greater than -2, the membership degree of a to negative very large is 1; accordingly, the triangular function value of negative very large is set as [-2.5, -2, -0.7]; (2) When a is lower than -1.2 or higher than -0.3, the membership degree of a to negative large is 0; when a is equal to -0.7, the membership degree of a to negative large is 1; accordingly, the triangular function value of negative large is set as [-1.2, -0.7, -0.3]; (3) When a is lower than -0.7 or higher than -0.1, the membership degree of a to negative medium is 0, when a is -0.3, the membership degree of a to negative medium is 1; accordingly, the triangular function value of negative medium is set as [-0.7, -0.3, -0.1]; (4) When a is lower than -0.3 or higher than 0, the membership degree of a to negative small is 0, when a is -0.1, the membership degree of a to negative small is 1; accordingly, the triangular function value of negative small is set as [-0.3, -0.1, 0]; (5) When a is lower than -0.1 or higher than 0.1, the membership degree of a to zero is 0, when a is 0, the membership degree of a to zero is 1; accordingly, the triangular function value of zero is set as [-0.1, 0, 0.1]; (6) When a is lower than 0 or higher than 0.2, the membership degree of a to positive small is 0, when a is 0.1, the membership degree of a to positive small is 1; accordingly, the triangular function value of positive small is set as [0, 0.1, 0.2]; (7) When a is lower than 0.1 or higher than 0.5, the membership degree of a to positive medium is 0, when a is 0.2, the membership degree of a to positive medium is 1; accordingly, the triangular function value of positive medium is set as [0.1, 0.2, 0.5]; (8) When a is lower than 0.2 or higher than 0.9, the membership degree of a to positive large is 0, when a is 0.5, the membership degree of a to positive large is 1; accordingly, the triangular function value of positive large is set as [0.2, 0.5, 0.9]. (9) When a is lower than 0.5 or higher than 1.5, a is 0 for positive very large membership, and when a is 0.9, a is 1 for positive very large membership; accordingly, the positive very large triangular function is set as [0.5, 0.9, 1.5].

7. The adaptive cruise decision method of claim 2, wherein, In step 6, after action b is performed, the discretized state variable model is updated as follows: where is the action of the second vehicle; and are discrete state quantities of the second vehicle's speed at different times, respectively; is the action of the first vehicle; and are discrete state quantities of the second vehicle's speed at different times, respectively; is a discrete state quantity of the actual distance between the two vehicles; τ = 1.

8. The adaptive cruise decision method of claim 7, wherein, In step 6, after action b is performed, the reward value r obtained is calculated as follows: r＝r g +r p ； wherein r g is the target reward value, r g rewards or punishes whether the target is reached in the adaptive cruise task; r g is set as follows: r p r is a process reward value p reward or penalize whether the adaptive cruise process is tending towards a better state; r p Set as follows: wherein x real is the actual distance between the two vehicles; and x safe is the safe distance between the two vehicles.

9. The adaptive cruise decision method of claim 8, wherein, In step 6, after action b is performed, the next time state s' obtained is set as follows: where v host is the speed of the second vehicle, x safe is the safety distance between the two vehicles.

Images