End-to-end automatic driving decision-making system and method based on chain thinking and medium

Abstract

The invention discloses an end-to-end automatic driving decision-making system and method based on chain thinking and a medium, and the system comprises a multi-task joint output module which obtains multi-source data, processes the multi-source data based on an attention mechanism, and obtains classification and regression data; the thinking chain step-by-step decision-making module is used for carrying out chain thinking step-by-step decision-making on the driving action of the automatic driving vehicle according to the classification and regression data to obtain a target vehicle speed; the control module is used for controlling the driving action of the vehicle according to the target vehicle speed and the own vehicle track waypoint; the decision-making process visualization module is used for carrying out partition visualization on the decision-making process; the multi-task joint output module is combined with an attention mechanism to collect multi-dimensional information of an automobile, the thinking chain step-by-step decision-making module calculates the target automobile speed step by step, the control module decouples transverse and longitudinal control according to the target automobile speed, the decision-making visualization module visualizes the chain type decision-making process, opacity of the decision-making process is reduced, and the decision-making efficiency is improved. And the interpretability of end-to-end driving is improved.

Description

Technical Field

[0001] The present invention relates to the field of vehicle subsystem joint control technology. Specifically, the present invention is applied to the field of autonomous driving, and in particular to an end-to-end autonomous driving decision-making system, method and medium based on chain thinking. Background Art

[0002] After years of development, autonomous driving technology has evolved from a rule-based, hierarchical modular architecture to an end-to-end (E2E) framework. Traditional modular architectures process environmental information step by step through independent modules such as perception, planning, and control. While these architectures offer advantages such as strong system interpretability and flexible module optimization, they also suffer from computational redundancy and low efficiency in multi-module collaboration. The end-to-end framework uses deep neural networks to directly map multimodal sensor inputs to vehicle control signals, significantly simplifying the system architecture and reducing the complexity of manual rule design. However, this technology has two core flaws in practical applications: (1) The separation between scene understanding and driving decision-making: The existing end-to-end model relies on an end-to-end black-box learning paradigm, and the causal relationship between the internal feature expression of the network and driving behavior is difficult to analyze. This implicit mapping mechanism makes it impossible for the system to establish a traceable association between scene features and driving decisions, resulting in a lack of technical support for accident tracing and responsibility definition, which restricts the reliability verification and regulatory adaptation of the system.

[0003] (2) Opacity of the decision-making process: The black box nature of the end-to-end model makes it impossible to generate a reasoning chain that conforms to human cognitive logic. Traditional deep learning models learn the mapping relationship between environmental features and control outputs through statistical correlation. Their decision-making basis is fundamentally different from the cognitive patterns of human drivers such as risk assessment and social interaction. This cognitive gap may cause the system to make counterintuitive decisions in complex scenarios, and human drivers and regulators find it difficult to understand its decision-making logic, which in turn raises questions about the safety and credibility of the system.

[0004] To address these challenges, the autonomous driving field urgently needs to build an explainable progressive decision-making framework. The ideal solution must meet three core requirements: 1) Establishing explicit feature representation from raw perception data to control output, breaking through the implicit mapping limitations of end-to-end models; 2) Ensure the coherence of decision logic through a hierarchical and progressive reasoning process, and achieve dynamic coordination of multi-level decision goals; 3) Generate decision-making basis that conforms to human cognitive habits, and enhance human-machine collaboration efficiency and regulatory acceptability. Summary of the Invention

[0005] The purpose of the present invention is to provide an end-to-end autonomous driving decision-making system, method and medium based on chain thinking in response to the above-mentioned problems in the prior art, thereby solving all or one of the above-mentioned problems in the prior art.

[0006] In order to solve the above technical problems, the specific technical solutions of the present invention are as follows: In one aspect, the present invention provides an end-to-end autonomous driving decision-making system based on chain thinking, comprising: A multi-task joint output module is used to obtain multi-source data about the autonomous driving vehicle, process the multi-source data based on the attention mechanism, and obtain corresponding classification and regression data; A thinking chain step-by-step decision module is used to make a step-by-step decision based on chain thinking on the driving action of the autonomous driving vehicle according to the classification and regression data, and obtain a target speed for the autonomous driving vehicle; a control module for controlling the vehicle's driving motion according to the target vehicle speed and the vehicle trajectory waypoints in the classification and regression data; The decision-making process visualization module is used to visualize the decision-making process partitions of the thought chain step-by-step decision-making module on the screen display device of the autonomous driving vehicle.

[0007] As an improved solution, the multi-source data includes: Ego vehicle speed, high-level commands, destination coordinates, and multi-frame video from multiple cameras on the autonomous vehicle; The classification and regression data include: Classification values ​​for collision risk, traffic lights, stop signs, and intersections; As well as, the regression value of the preceding vehicle state and the regression value of the predicted ego vehicle trajectory waypoint.

[0008] As an improved solution, the multi-task joint output module is specifically used to process the multiple frames of video based on the attention mechanism to obtain corresponding video tags; add position codes to the video tags and input them into the encoder of the attention mechanism to generate corresponding key-value pairs.

[0009] As an improved solution, the multi-task joint output module is further configured to define multiple queries, input the multiple queries and the key-value pairs into a decoder of the attention mechanism, obtain classification and regression labels corresponding to the multiple queries, and use the classification and regression labels as the classification and regression data; The multiple queries include: collision risk query, traffic light query, parking instruction query, intersection query, preceding vehicle status query and own vehicle trajectory waypoint query.

[0010] As an improved solution, the regression data corresponding to the collision risk query is a collision risk mark, and the categories of the collision risk mark include: collision risk exists and collision risk does not exist; The regression data corresponding to the traffic light query is a traffic light mark, and the categories of the traffic light mark include: the presence of a red light and the presence of a non-red light; The regression data corresponding to the parking instruction query is a parking instruction mark, and the categories of the parking instruction mark include: the presence of a parking instruction sign and the absence of a parking instruction sign; The regression data corresponding to the intersection query is an intersection mark, and the categories of the intersection mark include: located at the intersection and not located at the intersection; The regression data corresponding to the preceding vehicle status query includes: the preceding vehicle speed regressed through a linear layer, and the relative distance between the preceding vehicle and the ego vehicle regressed through a linear layer and a rectified linear unit activation layer; The regression data corresponding to the ego vehicle trajectory waypoint query includes: future trajectory waypoints consisting of six future waypoints with fixed distance intervals, and additional information consisting of the ego vehicle's current speed, high-level instructions, and destination coordinates.

[0011] As an improved solution, the thought chain step-by-step decision module is further configured to: execute a first decision process of a target speed based on collision risk re-judgment according to the collision risk mark to obtain a first target speed or a first continuing decision mark; The thought chain step-by-step decision module is further configured to: in response to the first continue decision mark, execute a second target speed decision process based on exponential decay calculation according to the traffic light mark and the stop sign to obtain a second target speed or a second continue decision mark; The thought chain step-by-step decision module is further configured to: in response to the second continuing decision indicator, execute a third target speed decision process based on a finite state machine according to the regression data corresponding to the preceding vehicle state query to obtain a third target speed; The thought chain step-by-step decision module is further configured to: use the first target speed, the second target speed, or the third target speed as the target vehicle speed according to the progress of the decision-making process.

[0012] As an improved solution, the control module is further configured to input the direction angle of the first waypoint of the future trajectory waypoint clock and the current heading angle of the autonomous driving vehicle into a lateral PID controller, and input the target vehicle speed into a longitudinal PID controller, and control the underlying vehicle actions according to the outputs of the lateral PID controller and the longitudinal PID controller.

[0013] As an improved solution, the decision-making process visualization module is specifically used to create a visualization interface on the screen display device of the autonomous driving vehicle, display the driving decision status in the top area of ​​the visualization interface, display the step-by-step decision results in the step-by-step decision-making process in the middle area of ​​the visualization interface, and use a highlighted flowchart in the bottom area of ​​the visualization interface to display the decision logic of the thinking chain step-by-step decision module, and trigger warnings for dangerous scenarios in the decision logic.

[0014] On the other hand, the present invention also provides an end-to-end autonomous driving decision-making method based on chain thinking, comprising the following steps: Acquire multi-source data about the autonomous driving vehicle, process the multi-source data based on an attention mechanism, and obtain corresponding classification and regression data; Based on the classification and regression data, a step-by-step decision is made on the driving action of the autonomous vehicle based on chain thinking to obtain a target speed of the autonomous vehicle; Controlling the vehicle's driving motion according to the target vehicle speed and the vehicle trajectory waypoints in the classification and regression data; The decision-making process of the step-by-step decision is partitioned and visualized on a screen display device of the autonomous driving vehicle.

[0015] On the other hand, the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the end-to-end autonomous driving decision-making method based on chain thinking are implemented.

[0016] The beneficial effects of the technical solution of the present invention are: 1. This system uses a multi-task joint output module combined with an attention mechanism to collect multi-dimensional information about the vehicle as input data for the subsequent thinking chain step-by-step decision-making module. The thinking chain step-by-step decision-making module calculates the target vehicle speed step by step according to the input data, and the control module decouples the horizontal and vertical controls according to the calculated target vehicle speed. The decision visualization module visualizes the chain decision-making process of the thinking chain step-by-step decision-making module based on a screen display method. Ultimately, this system reduces the vehicle-side demand for large models, reduces the opacity of the decision-making process, improves the interpretability of end-to-end driving, ensures the driving safety and reliability of autonomous vehicles, ensures the traceability of key data, and facilitates the safe maintenance of autonomous vehicles.

[0017] 2. The end-to-end autonomous driving decision-making method based on chain thinking described in the present invention can call system modules in an orderly manner, thereby realizing the system logic of the end-to-end autonomous driving decision-making system based on chain thinking described in the present invention.

[0018] 3. The computer-readable storage medium described in the present invention can enable the guidance system modules to cooperate, thereby realizing the end-to-end autonomous driving decision-making method based on chain thinking described in the present invention, and the computer-readable storage medium described in the present invention also effectively improves the operability of the end-to-end autonomous driving decision-making method based on chain thinking. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0020] Figure 1 This is a schematic diagram of the architecture of the end-to-end autonomous driving decision system based on chain thinking according to Example 1 of the present invention; Figure 2 This is a detailed architectural diagram of the end-to-end autonomous driving decision-making system based on chain thinking as described in Example 1 of the present invention; Figure 3 This is a decision logic diagram of the thinking chain step-by-step decision module in the end-to-end autonomous driving decision system based on chain thinking according to Example 1 of the present invention; Figure 4 This is a schematic diagram of a visualization interface of the decision-making process of the end-to-end autonomous driving decision-making system based on chain thinking in a certain driving scenario according to Example 1 of the present invention; Figure 5 This is a flow chart of the end-to-end autonomous driving decision-making method based on chain thinking described in Example 2 of the present invention. DETAILED DESCRIPTION

[0021] The preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making a clearer and more precise definition of the protection scope of the present invention.

[0022] In the description of the present invention, it should be noted that the embodiments described in the present invention are only part of the embodiments of the present invention, rather than all of the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative work are within the scope of protection of the present invention.

[0023] The terms "first," "second," and the like in the specification and claims herein and in the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate so that the embodiments of the present invention described herein can be implemented in orders other than those illustrated or described herein. In addition, the terms "including" and "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or device comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such process, method, product, or device.

[0024] In the description of the present invention, it should be noted that in the early stage of training of this system, we provide the model with a higher level of learning signals in the data collection stage based on the "expert strategy"; first, in the data generation stage, the expert strategy uses the API to obtain the road and lane IDs controlled by each traffic light; when the self-vehicle is driving according to the preset left-turn route, although there is a red light for going straight ahead, the expert strategy will identify the traffic light related to the self-vehicle's driving as a left-turn green light through the API based on the planned path of the vehicle, even if the straight-ahead light is a red light; therefore, in this frame of data, the "traffic light status" we marked is "green light" instead of "red light"; therefore, during training, the model does not learn isolated image information, but "in the left-turn task, facing the red light". For such a visual scene, the corresponding traffic signal is green. This natural filtering and labeling process enables the model to distinguish which traffic light is for its own driving task and learn the correct behavior. Therefore, this application does not simply train the model to identify "whether there is a red light." Therefore, the regression labels given in this application are all recognized result quantities, not random quantities that require decision-making. In the step-by-step decision-making process of this application, the early expert strategy has implicitly considered the actual application scenario conditions, and the final decision is based on the returned regression value. Therefore, it should also be noted that the main technical purpose and innovation of this application lies in the step-by-step decision-making and visualization process based on the thinking chain, rather than how to perceive the environment or detect the target.

[0025] Example 1: This example provides an end-to-end autonomous driving decision-making system based on chain thinking. Figures 1 to 4 As shown, including: (1) A multi-task joint output module is used to obtain environmental perception information, vehicle speed, high-level instructions, and navigation point information as input data, and input the input data into an end-to-end autonomous driving model composed of an attention mechanism model architecture. The multi-head network structure based on the attention mechanism model outputs the classification values ​​and regression values ​​required by the thinking chain.

[0026] (1.1) Input data specifically includes: vehicle speed, high-level instructions, destination coordinates, and multiple frames of video from multiple cameras; Among them, the multiple cameras include: a front camera, a left front camera, a right front camera, a rear camera, a left rear camera and a right rear camera, a total of six, and all six cameras are used to collect continuous frames from various perspectives.

[0027] (1.2) Output data specifically includes: classification values ​​for collision risk, traffic lights, stop signs, and intersections, and regression values ​​for the preceding vehicle status and predicted ego vehicle trajectory waypoints; (1.3) Specifically, the multi-task joint output module inputs the video data acquired by multiple cameras into the video attention mechanism model to obtain video tags. The video tags are then encoded with positions and input into the encoder of the attention mechanism model to generate corresponding key-value pairs. (1.4) Specifically, the multi-task joint output module defines multiple queries, namely: collision risk query, traffic light query, stop sign query, intersection query, preceding vehicle status query, and vehicle trajectory waypoint query; the multi-task joint output module inputs the defined queries and key-value pairs into the decoder of the attention mechanism model to obtain the classification and regression labels corresponding to each query.

[0028] Among them, the regression labels corresponding to collision risk query, traffic light query, stop sign query and intersection query are: collision risk label, traffic light label, stop sign label and intersection label; Collision risk marker, used to predict whether there is a potential collision risk between the vehicle and other vehicles, pedestrians, or other objects; the corresponding categories include: collision risk exists and no collision risk exists; Traffic light markers, used to determine whether the vehicle should wait for a red light; the corresponding categories include: red light present and non-red light; Parking sign, used to determine whether the vehicle can park in the area; the corresponding categories include: parking sign exists and parking sign does not exist; Intersection markings, used to determine whether the vehicle should be speed-limited at an intersection; the corresponding categories include: at an intersection and not at an intersection; The representations of the above four classification tasks are similar, so a unified representation is adopted as follows: ; Where, is the logical value of category 0, output by the corresponding prediction head; is the logical value of category 1; is the corresponding probability value, is the predicted category; When this formula is used in collision risk classification, Indicates a risk of collision. Indicates that there is no risk of collision; When this formula is used in traffic light classification,

[0029] When this formula is used in parking indication classification, Indicates the presence of a parking sign. Indicates that there is no parking sign; When this formula is used in intersection classification, Indicates that it is at an intersection. Indicates that you are not at an intersection.

[0030] Among them, the regression value corresponding to the state of the preceding vehicle includes: the speed of the preceding vehicle through linear layer regression , and the relative distance between the preceding vehicle and the vehicle regressed through the linear layer and the rectified linear unit activation layer

[0031]

[0032] Where, is the predicted speed of the preceding vehicle; is the weight vector of the velocity regression head; The labels for the preceding vehicle state prediction task extracted by the decoder; is the bias vector of the velocity regression head; is the predicted relative distance between the ego vehicle and the preceding vehicle; is the rectified linear unit activation function; is the weight vector of the relative distance regression head; is the bias vector of the relative distance regression head.

[0033] The predicted ego vehicle trajectory waypoints are obtained through gated recurrent unit network regression. The ego vehicle trajectory waypoints consist of six future waypoints with fixed distance intervals, and the current speed of the ego vehicle, high-level commands, and destination coordinates are used as additional inputs. The ego vehicle trajectory waypoints are decoupled from the velocity and expressed in (x, y) coordinate form to guide the ego vehicle's steering. The corresponding formula is as follows: ; Where, , represents six future vehicle trajectory waypoints; represents a gated recurrent unit network; is the initial hidden state; The tokens for trajectory prediction tasks extracted for the decoder; is the current speed of the vehicle, For high-level instructions, is the destination coordinate (x, y).

[0034] (2) The thinking chain step-by-step decision module is used to build an explainable autonomous driving decision framework based on the classification values ​​and regression values ​​obtained above, and calculate the final target speed of the vehicle based on this explainable autonomous driving decision framework. Figure 3 As shown, combined with the attached Figure 3 ,The specific decision-making process of the thinking chain step-by-step decision-making module is as follows: (2.1) The thought chain step-by-step decision module first performs the first decision process of the target speed based on the classification results of the collision risk prediction head: If the collision risk is classified as existing, the target speed is set to 0; If the collision risk is classified as no collision risk, the trajectory prediction module is called for further judgment: the trajectory prediction module assumes that the other vehicle is a two-wheeled bicycle model and maintains its original state (for example, if the original state is moving in a straight line, it continues to move in a straight line at a constant speed; if the original state is stationary, it continues to remain stationary). If the trajectory of the self-vehicle and the trajectory of the other vehicle have overlapping trajectory points in the next 6 frames, it is determined that there is a collision risk and the target speed is set to 0. If this is not the case (that is, the first continuation decision indicator is present), further decisions are made based on traffic lights and stop signs.

[0035] (2.2) The Thinking Chain Step-by-Step Decision Module performs the second decision process for the target speed based on the classification results of the traffic light and stop sign prediction heads: If the traffic light and stop sign are classified as a red light or a stop sign nearby, the target speed is set to 0; If the traffic light and stop sign are classified as a red light or a stop sign in the distance, the target speed is set to an exponential decay speed based on time collision, as shown below: ; Where, is the target speed; is the vehicle speed; is the attenuation coefficient; is the collision time; It is the distance from the vehicle to the lower limit of the red light or stop sign, that is, the distance from the vehicle to the red light or stop sign minus 30 meters (set according to specific circumstances).

[0036] If the traffic light and stop sign are classified as no red light or no stop sign (i.e., there is a second continuing decision indicator), further decision is made based on the regression value of the preceding vehicle state.

[0037] (2.3) The thinking chain step-by-step decision module performs the third decision process of the target speed based on the regression value of the preceding vehicle state: In response to the presence of a preceding vehicle, <6m or <15m and When the target speed is less than 4 m / s, the finite state machine is used to adjust the target speed, where the states include 𝒮 = {follow the vehicle, slow down, brake}; otherwise, the target speed is adjusted based on the time to collision. In response to the absence of a preceding vehicle, the target vehicle speed is set to the maximum speed limit.

[0038] The corresponding state transition conditions are as follows: ; In addition, to avoid state jitter, add a hysteresis interval (for example, it takes >7m, etc.) for damping treatment; In addition, speed limits are set based on whether the vehicle is turning or at an intersection. If the vehicle needs to turn or is at an intersection, the target vehicle speed is set to 5 m / s. Otherwise, the target vehicle speed is set to the maximum speed limit of the road. Based on the above three decision-making processes, the final target vehicle speed can be obtained.

[0039] (3) The control module is used to input the predicted ego-vehicle trajectory waypoints and the target speed generated above into the lateral PID controller and the longitudinal PID controller respectively to realize the control of the vehicle's underlying motion.

[0040] (3.1) The heading angle of the first waypoint in the ego-vehicle trajectory and the current heading angle are input into the lateral PID controller. The resulting output is the steering wheel angle, which has a value between -1 and 1. Here, -1 to 0 represents a left turn, and 0 to 1 represents a right turn. Specifically, the values ​​are as follows: ; Where, It is the steering wheel angle signal; is the direction error; is the proportionality coefficient; is the differential coefficient; is the integration coefficient; is the direction angle of the first waypoint; is the current heading angle of the vehicle.

[0041] (3.2) The target speed and the current vehicle speed are input into the longitudinal PID controller. The resulting output is the throttle or brake signal, with corresponding values ​​ranging from -1 to 1. Here, -1 to 0 represents the brake signal, and 0 to 1 represents the throttle signal. Specifically, they are represented as follows: ; Where, For throttle or brake signal; is the speed error; is the proportionality coefficient; is the differential coefficient; is the integration coefficient.

[0042] (4) Decision-making process visualization module, which is used to display the above-mentioned control signals and calculation and decision-making processes on the screen, highlight the current driving decision status at the top of the visualization interface, display key parameters in the middle of the visualization interface, and use a highlighted flow chart at the bottom of the visualization interface to display the real-time thinking chain judgment process, and trigger red or yellow warnings for dangerous scenes.

[0043] (4.1) The decision-making process visualization module sets a top status bar and uses the top status bar to provide real-time feedback on the current highest priority decision, which is expressed as: 𝒟 = {emergency braking, braking, turn left, turn right, follow the vehicle in front, slow down, accelerate, drive at the speed limit}; Among them, the decision process visualization module uses relevant icons to enhance visualization (such as a red warning icon is used to indicate emergency braking status).

[0044] (4.2) The decision process visualization module sets up a central parameter area and uses the central parameter area to display the key dynamic parameters that trigger the current decision, expressed as: 𝒫 = {collision risk, traffic light, stop sign, preceding vehicle status, control command}; The decision-making process visualization module highlights dynamic parameters related to the decision to enhance interpretability (for example, during emergency braking, the distance to the preceding vehicle and the control command are highlighted). The specific representation is as follows: ; Where, Indicates highlighting; , for the specific highest priority decision; , which are specific key dynamic parameters.

[0045] (4.3) The decision-making process visualization module sets up a bottom flowchart based on a fixed chain thinking template and pre-sets a fixed language template; the decision-making process visualization module uses the bottom flowchart to cover all elements of the decision-making process of the above-mentioned thinking chain step-by-step decision-making module, and uses a fill-in method to fill the specific decision-making process into the fixed language template.

[0046] As an example verification of this implementation, CARLA closed-loop simulation is used to verify the explainability of this system for end-to-end autonomous driving, as follows: The simulation scenario is set to a driving scenario with no red lights, no stop signs, no collisions, and other vehicles in the front lane. The visual decision-making process under the corresponding driving scenario is obtained as follows: Figure 4 ; Among them, the highest priority decision in the top status bar is braking, and a warning icon is used to enhance the expressiveness; The parameters in the middle, from top to bottom, are no collision risk, no traffic lights, no stop signs, status of the preceding vehicle, and control instructions. Among them, braking is related to the status of the vehicle in front and the control instructions, so it is highlighted; the thinking chain is shown at the bottom, and the content in brackets is dynamically updated according to the specific driving scenario.

[0047] Based on the above simulation results, it can be shown that this system provides a complete and clear thought chain explanation of the end-to-end autonomous driving decision-making process, enhancing the traceability and reliability of end-to-end autonomous driving decision-making data.

[0048] It should be noted that the above examples are only for explaining the present invention and are not intended to limit the scope of protection of the present invention.

[0049] Example 2: This example is based on the same inventive concept as the end-to-end autonomous driving decision system based on chain thinking described in Example 1, and provides an end-to-end autonomous driving decision method based on chain thinking, such as Figure 5 As shown, the following steps are included: S100, obtaining multi-source data about the autonomous driving vehicle, processing the multi-source data based on an attention mechanism, and obtaining corresponding classification and regression data; S200, performing a step-by-step decision on the driving action of the autonomous driving vehicle based on the classification and regression data based on chain thinking to obtain a target speed for the autonomous driving vehicle; S300, controlling the vehicle's driving motion according to the target vehicle speed and the vehicle trajectory waypoints in the classification and regression data; S400: Visualize the decision-making process of the step-by-step decision-making in partitions on a display device of the autonomous driving vehicle.

[0050] Embodiment 3: This embodiment provides a computer-readable storage medium, including: The storage medium is used to store computer software instructions used to implement the end-to-end autonomous driving decision-making method based on chain thinking described in the above-mentioned Example 2, which includes a program for executing the above-mentioned program set for the end-to-end autonomous driving decision-making method based on chain thinking; specifically, the executable program can be built into the end-to-end autonomous driving decision-making system based on chain thinking described in Example 1, so that the end-to-end autonomous driving decision-making system based on chain thinking can implement the end-to-end autonomous driving decision-making method based on chain thinking described in Example 2 by executing the built-in executable program.

[0051] In addition, the computer-readable storage medium of this embodiment may adopt any combination of one or more computer-readable storage media, wherein the computer-readable storage medium includes electrical, optical, electromagnetic, infrared or semiconductor systems, devices or components, or any combination thereof.

[0052] Different from the existing technology, the present application adopts an end-to-end autonomous driving decision-making system, method and medium based on chain thinking. The target vehicle speed can be calculated step by step according to the input data through the thinking chain step-by-step decision module, the horizontal and vertical controls can be decoupled according to the calculated target vehicle speed through the control module, and the chain decision-making process of the thinking chain step-by-step decision module can be visualized based on the screen display method through the decision visualization module. Ultimately, this system reduces the vehicle-side demand for large models, reduces the opacity of the decision-making process, improves the interpretability of end-to-end driving, ensures the driving safety and reliability of autonomous driving vehicles, ensures the traceability of key data, and facilitates the safe maintenance of autonomous driving vehicles.

[0053] It should be understood that in the various embodiments of this document, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this document.

[0054] It should also be understood that in the embodiments herein, the term "and / or" merely describes an association between associated objects, indicating that three possible relationships exist. For example, "A and / or B" could represent: A alone, A and B simultaneously, or B alone. Furthermore, the character " / " in this document generally indicates an "or" relationship between the associated objects.

[0055] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the composition and steps of each example according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this document.

[0056] Those skilled in the art will clearly understand that, for the sake of convenience and brevity of description, the specific logical process of the method described above can refer to the corresponding working processes of the systems, devices and units in the aforementioned method embodiments, and will not be repeated here.

[0057] In the several embodiments provided herein, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices, or units, or can be an electrical, mechanical, or other form of connection.

[0058] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments herein.

[0059] In addition, the functional units in the various embodiments herein may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0060] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this article is essentially or the part that contributes to the existing technology, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of this article. The aforementioned storage medium includes: various media that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0061] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention's description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An end-to-end autonomous driving decision-making system based on chain thinking, characterized by: include: A multi-task joint output module is used to obtain multi-source data about the autonomous driving vehicle, process the multi-source data based on the attention mechanism, and obtain corresponding classification and regression data; A thinking chain step-by-step decision module is used to make a step-by-step decision based on chain thinking on the driving action of the autonomous driving vehicle according to the classification and regression data, and obtain a target speed for the autonomous driving vehicle; a control module for controlling the vehicle's driving motion according to the target vehicle speed and the vehicle trajectory waypoints in the classification and regression data; The decision-making process visualization module is used to visualize the decision-making process partitions of the thought chain step-by-step decision-making module on the screen display device of the autonomous driving vehicle.

2. The end-to-end autonomous driving decision-making system based on chain thinking according to claim 1 is characterized by: The multi-source data includes: Ego vehicle speed, high-level commands, destination coordinates, and multi-frame video from multiple cameras on the autonomous vehicle; The classification and regression data include: Classification values ​​for collision risk, traffic lights, stop signs, and intersections; As well as, the regression value of the preceding vehicle state and the regression value of the predicted ego vehicle trajectory waypoint.

3. The end-to-end autonomous driving decision-making system based on chain thinking according to claim 2 is characterized by: The multi-task joint output module is specifically used to process the multiple frames of video based on the attention mechanism to obtain corresponding video tags; add position codes to the video tags and input them into the encoder of the attention mechanism to generate corresponding key-value pairs.

4. The end-to-end autonomous driving decision-making system based on chain thinking according to claim 3 is characterized by: The multi-task joint output module is further configured to define multiple queries, input the multiple queries and the key-value pairs into the decoder of the attention mechanism, obtain classification and regression labels corresponding to the multiple queries, and use the classification and regression labels as the classification and regression data; The multiple queries include: collision risk query, traffic light query, parking instruction query, intersection query, preceding vehicle status query and own vehicle trajectory waypoint query.

5. The end-to-end autonomous driving decision-making system based on chain thinking according to claim 4 is characterized by: The regression data corresponding to the collision risk query is a collision risk mark, and the categories of the collision risk mark include: collision risk exists and collision risk does not exist; The regression data corresponding to the traffic light query is a traffic light mark, and the categories of the traffic light mark include: the presence of a red light and the presence of a non-red light; The regression data corresponding to the parking instruction query is a parking instruction mark, and the categories of the parking instruction mark include: the presence of a parking instruction sign and the absence of a parking instruction sign; The regression data corresponding to the intersection query is an intersection mark, and the categories of the intersection mark include: located at the intersection and not located at the intersection; The regression data corresponding to the preceding vehicle status query includes: the preceding vehicle speed regressed through a linear layer, and the relative distance between the preceding vehicle and the ego vehicle regressed through a linear layer and a rectified linear unit activation layer; The regression data corresponding to the ego vehicle trajectory waypoint query includes: future trajectory waypoints consisting of six future waypoints with fixed distance intervals, and additional information consisting of the ego vehicle's current speed, high-level instructions, and destination coordinates.

6. The end-to-end autonomous driving decision-making system based on chain thinking according to claim 5 is characterized by: The thought chain step-by-step decision module is further configured to: execute a first decision process of a target speed based on collision risk re-judgment according to the collision risk mark to obtain a first target speed or a first continuing decision mark; The thought chain step-by-step decision module is further configured to: in response to the first continue decision mark, execute a second target speed decision process based on exponential decay calculation according to the traffic light mark and the stop sign to obtain a second target speed or a second continue decision mark; The thought chain step-by-step decision module is further configured to: in response to the second continuing decision indicator, execute a third target speed decision process based on a finite state machine according to the regression data corresponding to the preceding vehicle state query to obtain a third target speed; The thought chain step-by-step decision module is further configured to: use the first target speed, the second target speed, or the third target speed as the target vehicle speed according to the progress of the decision-making process.

7. The end-to-end autonomous driving decision-making system based on chain thinking according to claim 5 is characterized by: The control module is further configured to input the direction angle of the first waypoint of the future trajectory waypoint clock and the current heading angle of the autonomous driving vehicle into a lateral PID controller, input the target vehicle speed into a longitudinal PID controller, and control the underlying vehicle actions according to the outputs of the lateral PID controller and the longitudinal PID controller.

8. The end-to-end autonomous driving decision-making system based on chain thinking according to claim 1 is characterized by: The decision-making process visualization module is specifically used to create a visualization interface on the screen display device of the autonomous driving vehicle, display the driving decision status in the top area of ​​the visualization interface, display the step-by-step decision results in the step-by-step decision-making process in the middle area of ​​the visualization interface, and display the decision logic of the thinking chain step-by-step decision module in the bottom area of ​​the visualization interface, and trigger warnings for dangerous scenarios in the decision logic.

9. An end-to-end autonomous driving decision-making method based on chain thinking, characterized in that: The following steps are involved: Acquire multi-source data about the autonomous driving vehicle, process the multi-source data based on an attention mechanism, and obtain corresponding classification and regression data; Based on the classification and regression data, a step-by-step decision is made on the driving action of the autonomous vehicle based on chain thinking to obtain a target speed of the autonomous vehicle; Controlling the vehicle's driving motion according to the target vehicle speed and the vehicle trajectory waypoints in the classification and regression data; The decision-making process of the step-by-step decision is partitioned and visualized on a screen display device of the autonomous driving vehicle.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the end-to-end autonomous driving decision-making method based on chain thinking described in claim 9.

Images