Automatic driving planning method and device, electronic equipment and storage medium
The multi-layer LSTM-CONCAT neural network is used to carry out autonomous driving path planning, which solves the problem that the existing technology cannot adapt to complex lane lines, and achieves better vehicle trajectory planning and safety.
Patent Information
- Application Number
- CN202411831150.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-12
- Publication Date
- 2025-03-21
AI Technical Summary
The existing autonomous driving planning methods cannot effectively adapt to the situation of complex lane lines, resulting in unsatisfactory vehicle trajectory planning, rapid steering wheel, and driving direction does not meet the driver's expectations.
The multi-layer LSTM-CONCAT neural network is used to plan the path of ramps and complex lanes, and the road environment changes and complex scenarios are processed through the breadth LSTM layer and the depth LSTM layer, and the road environment changes and complex scenarios are fused in the CONCAT layer to output the planning state results.
It improves the generalization adaptability of vehicle trajectory planning, and can effectively handle complex scenarios such as lane diversion, convergence, and widening to ensure the certainty and safety of the target path.
Smart Images
Figure CN119659671A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of autonomous driving technology, and more specifically, to an autonomous driving planning method, device, electronic device and storage medium. Background Art
[0002] When the left and right lanes of the main lane where the vehicle is located are clear, the autonomous vehicle will follow the center line of the two lanes as the target path. However, on urban roads, there are often irregular lane lines such as lane divergence, merging, lane widening, and extra-wide lanes; there are also ramps, only one-side lane lines, only one-side curbs, or a mixture of lane lines and curbs. In these cases, there are no two obvious main lane lines to follow. If the result of following the center line of the lane line is not ideal, there will be problems such as sudden steering and driving direction not meeting the driver's expectations. Summary of the invention
[0003] The purpose of the embodiments of the present application is to provide an autonomous driving planning method, device, electronic device and storage medium, which utilize a multi-layer LSTM-CONCAT neural network to perform path planning for ramps and complex lanes, thereby improving the generalization adaptability of vehicle trajectory planning and solving the problem that existing planning methods cannot adapt to complex lane lines.
[0004] The present application provides an automatic driving planning method, the method comprising:
[0005] Acquire sensory data;
[0006] Inputting the perception data into a wide LSTM layer and a deep LSTM layer of a neural network;
[0007] The output results of the breadth LSTM layer and the depth LSTM layer are fused in the CONCAT layer, and the planning state results are output through the fully connected layer and the SOFTMAX layer;
[0008] A target path is obtained according to the planning status result.
[0009] In the above implementation process, the breadth LSTM layer can be used to adjust the output of the planning state when the road environment changes, and the depth LSTM layer can be used to handle complex scenarios such as lane divergence, merging, and widening. Therefore, the multi-layer LSTM-CONCAT neural network can be used to perform path planning for ramps and complex lanes, improve the generalization adaptability of vehicle trajectory planning, and solve the problem that the existing planning method cannot adapt to complex lane lines.
[0010] Furthermore, the method further comprises:
[0011] Obtaining the confidence of the planning status result;
[0012] The confidence level of the planning status result is determined: if the current confidence level exceeds a preset threshold, the current planning status is updated to the planning status result; otherwise, the planning status at the previous moment is used.
[0013] In the above implementation process, the confidence level is used to judge the planning status results to ensure the certainty of the target path finally outputted.
[0014] Further, the obtaining of the confidence level of the planning status result includes:
[0015] The maximum value in the output vector of the SOFTMAX layer is obtained, and the planning state result corresponding to the maximum value is used as the final planning state. The size of the maximum value represents the confidence of the planning state.
[0016] In the above implementation process, the maximum value of the output vector is used to represent the confidence of the planning state, so as to determine the correctness and safety of the planning state.
[0017] Furthermore, the acquiring of the perception data includes:
[0018] The perception data sent by the perception fusion module is received, where the perception data includes left and right lane line data and left and right curb data, and the shapes of the lane line and the curb are represented by a cubic polynomial.
[0019] In the above implementation process, perception data is received to provide a data basis for subsequent path planning.
[0020] Further, the planning status result includes: planning failure, two-lane line following, single lane line following, single lane line type following, lane line and curb following, single curb following, and obtaining the target path according to the planning status result includes:
[0021] If the planning status result is two-lane lane following, the target path is the center line of the left and right lanes;
[0022] If the planning status result is single lane line following, the target path is to follow only one of the left or right lane lines, and the road width is the set value;
[0023] If the planning status result is single lane line following, the target path is to follow only one of the left and right lane lines, and the road width is the width of the left and right lane lines;
[0024] If the planning status result is lane line and curb following, the target path is to follow the lane line on one side and the curb on the other side;
[0025] If the planning status result is single curb following, the target path is to follow only one of the left and right curbs, and the road width is the set value.
[0026] In the above implementation process, the target path is determined according to the planning state results to achieve the purpose of autonomous driving planning.
[0027] The present application also provides an automatic driving planning device, the device comprising:
[0028] A data acquisition module, used to acquire perception data;
[0029] A data input module, used for inputting the perception data into a width LSTM layer and a depth LSTM layer of a neural network;
[0030] A fusion module is used to fuse the output results of the breadth LSTM layer and the depth LSTM layer in the CONCAT layer, and output the planning state results through the fully connected layer and the SOFTMAX layer;
[0031] The target path acquisition module is used to acquire the target path according to the planning state result.
[0032] In the above implementation process, the breadth LSTM layer can be used to adjust the output of the planning state when the road environment changes, and the depth LSTM layer can be used to handle complex scenarios such as lane divergence, merging, and widening. Therefore, the multi-layer LSTM-CONCAT neural network can be used to perform path planning for ramps and complex lanes, improve the generalization adaptability of vehicle trajectory planning, and solve the problem that the existing planning method cannot adapt to complex lane lines.
[0033] Furthermore, the device also includes:
[0034] A confidence acquisition module, used to obtain the confidence of the planning status result;
[0035] The confidence determination module is used to perform confidence determination on the planning status result: if the current confidence exceeds a preset threshold, the current planning status is updated to the planning status result; otherwise, the planning status at the previous moment is used.
[0036] In the above implementation process, the confidence level is used to judge the planning status results to ensure the certainty of the target path finally outputted.
[0037] Furthermore, the confidence acquisition module includes:
[0038] The vector value acquisition module is used to obtain the maximum value in the output vector of the SOFTMAX layer, and use the planning state result corresponding to the maximum value as the final planning state. The size of the maximum value represents the confidence of the planning state.
[0039] In the above implementation process, the maximum value of the output vector is used to represent the confidence of the planning state, so as to determine the correctness and safety of the planning state.
[0040] An embodiment of the present application also provides an electronic device, which includes a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the above-mentioned autonomous driving planning method.
[0041] An embodiment of the present application also provides a readable storage medium, in which computer program instructions are stored. When the computer program instructions are read and executed by a processor, the above-mentioned autonomous driving planning method is executed. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0043] Figure 1 A flowchart of an autonomous driving planning method provided in an embodiment of the present application;
[0044] Figure 2 A confidence determination flow chart provided in an embodiment of the present application;
[0045] Figure 3 A flowchart of autonomous driving planning using a multi-layer LSTM-CONCAT neural network provided in an embodiment of the present application;
[0046] Figure 4 A structural block diagram of an autonomous driving planning device provided in an embodiment of the present application;
[0047] Figure 5 A structural block diagram of another autonomous driving planning device provided in an embodiment of the present application.
[0048] icon:
[0049] 100 - data acquisition module; 200 - data input module; 300 - fusion module; 310 - confidence acquisition module; 311 - vector value acquisition module; 320 - confidence determination module; 400 - target path acquisition module. DETAILED DESCRIPTION
[0050] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.
[0051] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.
[0052] Please see Figure 1 , Figure 1 A flowchart of an autonomous driving planning method provided in an embodiment of the present application. The method specifically comprises the following steps:
[0053] Step S100: Acquire perception data;
[0054] Step S200: inputting the perception data into the breadth LSTM layer and the depth LSTM layer of the neural network;
[0055] Step S300: The output results of the breadth LSTM layer and the depth LSTM layer are fused in the CONCAT layer, and the planning state results are output through the fully connected layer and the SOFTMAX layer;
[0056] Step S400: Acquire a target path according to the planning status result.
[0057] The multi-layer LSTM-CONCAT neural network uses wide LSTM layers and deep LSTM layers, which have generalization ability and memory. It can effectively improve the ability of autonomous driving vehicles to handle ramps and complex urban roads, and while ensuring the network response speed, it can improve the ability to remember complex road conditions.
[0058] On the basis of the above embodiment, in order to ensure the stability of the planning state results output by the SOFTMAX layer, a confidence determination mechanism can be added to ensure the safety of the target path during driving and its consistency with the current road, such as Figure 2 As shown in FIG. 1 , it is a confidence determination flow chart, which specifically includes the following steps:
[0059] Step S310: obtaining the confidence level of the planning status result;
[0060] Specifically, the maximum value in the output vector of the SOFTMAX layer is obtained, and the planning state result corresponding to the maximum value is used as the final planning state, and the size of the maximum value represents the confidence of the planning state.
[0061] The output of the SOFTMAX layer is a 10*1 vector, which corresponds to the following 10 possible planning state results. The vector value is encoded using OneHot, that is, the sum of all numbers in the vector is 1. Among them, the largest value in the vector represents the decision choice of the current neural network output. The size of the maximum value represents the degree of certainty of the current choice, that is, the confidence. The vector values with vector subscripts 0-9 correspond to the following 10 planning state results. For example, if the subscript of the maximum value is 9, the output planning state is the 10th planning state: right edge following.
[0062] Step S320: Perform confidence determination on the planning status result: if the current confidence exceeds a preset threshold, update the current planning status to the planning status result; otherwise, continue to use the planning status at the previous moment.
[0063] To ensure the stability of the final output planning state, a confidence judgment mechanism is added after the output of the SOFTMAX layer. Only when the current confidence exceeds the preset threshold, the current planning state is updated to the output of the neural network, otherwise the planning state of the previous moment is used.
[0064] like Figure 3 As shown, it is a flow chart of autonomous driving planning using a multi-layer LSTM-CONCAT neural network, wherein the perception data in step S100 can receive perception data sent by a perception fusion module, and the perception data includes left and right lane line data, left and right curb data, and the vehicle speed, and the shapes of the lane lines and curbs are represented by cubic polynomials.
[0065] Specifically, the perception data is input from the sequence input layer to the breadth LSTM layer and depth LSTM layer of the neural network. The breadth LSTM layer is a single-layer LSTM network, and the depth LSTM layer is a three-layer LSTM network. The outputs of the breadth LSTM layer and the depth LSTM layer will be fused by the input value CONCAT layer. Finally, the planning state result is output through the fully connected layer and the SOFTMAX layer.
[0066] The number of hidden units in the breadth LSTM layer is larger than the number of hidden units in each single layer of the depth LSTM layer. Its function is to adjust the output of the planning state in time when the road environment changes rapidly. For example, when the main lane line changes from two lanes to only the left lane, the neural network can be led by the breadth LSTM layer to switch from two lanes to single lane.
[0067] The deep LSTM layer is used to handle complex working conditions such as lane divergence, merging, and widening. By utilizing the stacking of memory capabilities in the multi-layer LSTM network, the changes in road conditions over a longer period of time are learned to make better planning decisions under complex working conditions.
[0068] According to the working conditions that the autonomous driving vehicle may encounter on the road, the planning status results may include the following states: planning failure, two-lane line following, single-lane line following, single-lane line line shape following, lane line and curb following, single curb following, among which, planning failure means that the planning conditions are not met, then the planning will exit; and single-lane line following, single-lane line line shape following, lane line and curb following, single curb following are divided into left and right states (the target path below takes the left side as an example), so there are a total of 10 possible planning status results.
[0069] The specific process of determining the target path in step S400 is as follows:
[0070] Planning failure means that planning conditions are not met and the planning is withdrawn. Planning conditions are the following planning conditions.
[0071] If the planning status result is two-lane lane following, the target path is the center line of the left and right lanes;
[0072] The target path at this time can be expressed as:
[0073] y=((L3+R3) / 2)*x^3+((L2+R2) / 2)*x^2+((L1+R1) / 2)*x+((L0+R0) / 2);
[0074] Among them, the left lane line coefficients are from high to low: L3, L2, L1, L0; correspondingly, the right lane line coefficients are represented by R, which are R3, R2, R1, R0; the left curb coefficient is represented by Le, and the right curb coefficient is represented by Re.
[0075] If the planning status result is single lane line following, the target path is to follow only one of the left or right lane lines, and the road width is the set value;
[0076] For example, the road width can use the default value set. Taking the left lane line as an example, the target path can be expressed as:
[0077] y=L3*x^3+L2*x^2+L1*x+(L0+W / 2);
[0078] Wherein, W represents the set value.
[0079] If the planning status result is single lane line following, the target path is to follow only one of the left and right lane lines, and the road width is the width of the left and right lane lines;
[0080] For example, taking the left lane line as an example, the target path can be expressed as:
[0081] y=L3*x^3+L2*x^2+L1*x+((L0+R0) / 2).
[0082] If the planning status result is lane line and curb following, the target path is to follow the lane line on one side and the curb on the other side;
[0083] For example, taking the left lane line and the right road edge as an example, the target path can be expressed as:
[0084] y=((L3+Re3) / 2)*x^3+((L2+Re2) / 2)*x^2+((L1+Re1) / 2)*x+((L0+Re0) / 2).
[0085] If the planning status result is single curb following, the target path is to follow only one of the left and right curbs, and the road width is the set value.
[0086] For example, the road width can use the default value set. Taking the left side of the road as an example, the target path can be expressed as:
[0087] y=Le3*x^3+Le2*x^2+Le1*x+(Le0+W / 2).
[0088] Wherein, W represents the set value.
[0089] In addition, the perception data also includes the confidence and effective length of lane lines and curbs. While neural networks have the advantage of generalization, their output has a certain degree of uncertainty. For safety reasons, the following planning conditions are set:
[0090] When all lane lines and curb confidences are invalid, the planning status result must be in an invalid state;
[0091] When following two lanes or one lane, you need to ensure that the confidence of both the left and right lanes is valid, otherwise the planning status needs to be set to invalid.
[0092] When following a single lane or a single curb, you need to ensure that the confidence of the corresponding lane or curb is valid, otherwise the planning status needs to be set to invalid;
[0093] When following lane lines and curbs: ensure that the confidence of the corresponding lane lines and curbs is valid, otherwise the planning status needs to be set to invalid.
[0094] If the above conditions and confidence determination are met, the target path can be determined based on the above planning status results.
[0095] The autonomous driving planning method based on the multi-layer LSTM-CONCAT neural network of this application uses the generalization ability of the neural network to effectively improve the ability of autonomous driving vehicles to handle ramps and complex roads. The neural network uses a wide LSTM layer and a deep LSTM layer to ensure the network response speed while improving the ability to remember complex road conditions. At the same time, a confidence and safety backup mechanism is set to ensure the consistency and safety of the target path during driving.
[0096] The present application also provides an automatic driving planning device, which is applied to the automatic driving planning method described above. Figure 4 The figure is a structural block diagram of an automatic driving planning device, which includes but is not limited to:
[0097] A data acquisition module 100, used to acquire perception data;
[0098] Specifically, the perception data sent by the perception fusion module is received, where the perception data includes left and right lane line data and left and right roadside data, and the shapes of the lane lines and the roadside are represented by cubic polynomials.
[0099] A data input module 200, used to input the perception data into the breadth LSTM layer and the depth LSTM layer of the neural network;
[0100] A fusion module 300 is used to fuse the output results of the breadth LSTM layer and the depth LSTM layer in a CONCAT layer, and output the planning state results through a fully connected layer and a SOFTMAX layer;
[0101] The target path acquisition module 400 is used to acquire the target path according to the planning state result.
[0102] like Figure 5 As shown, it is a structural block diagram of another automatic driving planning device, and the device also includes:
[0103] A confidence acquisition module 310, used to obtain the confidence of the planning status result;
[0104] The confidence determination module 320 is used to perform confidence determination on the planning status result: if the current confidence exceeds a preset threshold, the current planning status is updated to the planning status result; otherwise, the planning status at the previous moment is used.
[0105] The confidence acquisition module 310 includes:
[0106] The vector value acquisition module 311 is used to obtain the maximum value in the output vector of the SOFTMAX layer, and use the planning state result corresponding to the maximum value as the final planning state. The size of the maximum value represents the confidence of the planning state.
[0107] The planning status results include: planning failure, two-lane line following, single lane line following, single lane line line type following, lane line and curb following, single curb following. The target path can be determined as:
[0108] If the planning status result is two-lane lane following, the target path is the center line of the left and right lanes;
[0109] If the planning status result is single lane line following, the target path is to follow only one of the left or right lane lines, and the road width is the set value;
[0110] If the planning status result is single lane line following, the target path is to follow only one of the left and right lane lines, and the road width is the width of the left and right lane lines;
[0111] If the planning status result is lane line and curb following, the target path is to follow the lane line on one side and the curb on the other side;
[0112] If the planning status result is single curb following, the target path is to follow only one of the left and right curbs, and the road width is the set value.
[0113] An embodiment of the present application also provides an electronic device, which includes a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the above-mentioned autonomous driving planning method.
[0114] An embodiment of the present application also provides a readable storage medium, in which computer program instructions are stored. When the computer program instructions are read and executed by a processor, the above-mentioned autonomous driving planning method is executed.
[0115] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.
[0116] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
[0117] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0118] The above description is only an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.
[0119] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
[0120] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
Claims
1. An automatic driving planning method, characterized in that: The method comprises: Acquire sensory data; Inputting the perception data into a wide LSTM layer and a deep LSTM layer of a neural network; The output results of the breadth LSTM layer and the depth LSTM layer are fused in the CONCAT layer, and the planning state results are output through the fully connected layer and the SOFTMAX layer; A target path is obtained according to the planning status result.
2. The automatic driving planning method according to claim 1, characterized in that: Before the step of acquiring the target path according to the planning state result, the method further includes: Obtaining the confidence of the planning status result; The confidence level of the planning status result is determined: if the current confidence level exceeds a preset threshold, the current planning status is updated to the planning status result; otherwise, the planning status at the previous moment is used.
3. The automatic driving planning method according to claim 2, characterized in that: The obtaining of the confidence level of the planning status result includes: The maximum value in the output vector of the SOFTMAX layer is obtained, and the planning state result corresponding to the maximum value is used as the final planning state. The size of the maximum value represents the confidence of the planning state.
4. The automatic driving planning method according to claim 1, characterized in that: The acquiring of the perception data comprises: The perception data sent by the perception fusion module is received, where the perception data includes left and right lane line data and left and right curb data, and the shapes of the lane line and the curb are represented by a cubic polynomial.
5. The method for autonomous driving planning according to claim 1, characterized in that: The planning status results include: planning failure, two-lane line following, single lane line following, single lane line line type following, lane line and curb following, single curb following, and obtaining the target path according to the planning status results includes: If the planning status result is two-lane lane following, the target path is the center line of the left and right lanes; If the planning status result is single lane line following, the target path is to follow only one of the left or right lane lines, and the road width is the set value; If the planning status result is single lane line following, the target path is to follow only one of the left and right lane lines, and the road width is the width of the left and right lane lines; If the planning status result is lane line and curb following, the target path is to follow the lane line on one side and the curb on the other side; If the planning status result is single curb following, the target path is to follow only one of the left and right curbs, and the road width is the set value.
6. An automatic driving planning device, characterized in that: The device comprises: A data acquisition module, used to acquire perception data; A data input module, used for inputting the perception data into a width LSTM layer and a depth LSTM layer of a neural network; A fusion module is used to fuse the output results of the breadth LSTM layer and the depth LSTM layer in the CONCAT layer, and output the planning state results through the fully connected layer and the SOFTMAX layer; The target path acquisition module is used to acquire the target path according to the planning state result.
7. The automatic driving planning device according to claim 6, characterized in that: The device also includes: A confidence acquisition module, used to obtain the confidence of the planning status result; The confidence determination module is used to perform confidence determination on the planning status result: if the current confidence exceeds a preset threshold, the current planning status is updated to the planning status result; otherwise, the planning status at the previous moment is used.
8. The automatic driving planning device according to claim 7, characterized in that: The confidence acquisition module comprises: The vector value acquisition module is used to obtain the maximum value in the output vector of the SOFTMAX layer, and use the planning state result corresponding to the maximum value as the final planning state. The size of the maximum value represents the confidence of the planning state.
9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the autonomous driving planning method according to any one of claims 1 to 5.
10. A readable storage medium, characterized in that: The readable storage medium stores computer program instructions, and when the computer program instructions are read and executed by a processor, the automatic driving planning method according to any one of claims 1 to 5 is executed.