Vehicle system
The vehicle system addresses the issue of autonomous vehicles disrupting traffic flow by using processing units to match speed with traffic flow and consider user intentions, thereby reducing congestion and accidents.
Patent Information
- Application Number
- PCT/JP2026/015551
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-05-09
- Filing Date
- 2026-04-17
- Publication Date
- 2026-11-12
AI Technical Summary
When vehicles are caused to travel autonomously, they may not match the traffic flow, leading to potential traffic congestion and accidents due to disturbances in the traffic flow.
A vehicle system that includes processing units to acquire external environment data, grasp traffic flow speed, and generate planned trajectories to match the vehicle's speed with the traffic flow, while also considering user intentions regarding speed limits.
This system reduces the likelihood of traffic disturbances by ensuring the vehicle's behavior aligns with traffic flow, minimizing disruptions and potential accidents.
Smart Images

Figure JP2026015551_12112026_PF_FP_ABST
Abstract
Description
Vehicle system Cross-reference to related applications
[0001] This application is based on Japanese Patent Application No. 2025-078974 filed on May 9, 2025, the entire content of which is incorporated herein by reference.
[0002] The disclosure of this specification relates to a technology for causing a vehicle to travel autonomously.
[0003] The method disclosed in Patent Document 1 is a method of learning a driver's driving style, setting a parameter set for each individual style, and automatically adjusting the speed of the vehicle based on the set.
[0004] German Patent Application Publication No. 102016205260 Specification
[0005] However, when a vehicle is caused to travel autonomously so as to reproduce an individual style, the behavior of the vehicle may not match the traffic flow on the road, and disturbance of the traffic flow may occur. There is a concern that the disturbance of the traffic flow may increase the possibility of occurrence of traffic troubles such as traffic congestion and accidents.
[0006] One of the objectives of the disclosure of this specification is to provide a vehicle system that reduces the possibility of occurrence of traffic troubles.
[0007] One of the aspects disclosed herein is a vehicle system that includes at least one processing unit and executes processing for causing a vehicle to travel autonomously, wherein the at least one processing unit is configured to execute acquiring data indicating an external environment of the vehicle, and grasping a speed of a traffic flow based on the data, and generating a planned trajectory for causing the vehicle to travel at a speed matching the speed of the traffic flow.
[0008] According to such an aspect, the speed of the traffic flow on the road on which the vehicle travels is grasped based on the data indicating the external environment, and the planned trajectory for causing the vehicle to travel at a speed matching the speed of the traffic flow is generated. By adopting the planned trajectory generated in this manner, it is possible to match the behavior of the vehicle with the traffic flow on the road, and the occurrence of disturbance in the traffic flow can be suppressed. As a result, it is possible to reduce the possibility of occurrence of traffic troubles.
[0009] Furthermore, one of the disclosed embodiments is a vehicle system comprising at least one processing unit that performs processing for autonomously driving a vehicle, wherein the at least one processing unit is configured to: confirm the user's intention regarding exceeding the speed limit; generate one or more planned trajectories for driving the vehicle; and, if the user's intention indicates prohibiting exceeding the speed limit, prohibit the adoption of a planned trajectory that exceeds the speed limit, and if the user's intention indicates allowing exceeding the speed limit, allow the adoption of a planned trajectory that exceeds the speed limit.
[0010] In this configuration, the decision to adopt or reject a generated planned trajectory that exceeds the speed limit is made based on the user's confirmation of their intent. For example, if the user indicates that exceeding the speed limit is acceptable, the adoption of the planned trajectory that exceeds the speed limit is permitted. Therefore, even if the speed of the traffic flow exceeds the speed limit, it becomes possible to control the vehicle in a way that minimizes disruption to the traffic flow. Consequently, the possibility of traffic disruptions can be reduced.
[0011] The symbols in parentheses included in the claims, etc., are illustrative examples illustrating the correspondence with the embodiments described later, and are not intended to limit the technical scope.
[0012] A diagram showing an example of the hardware configuration of the driving system. A diagram showing an example of the architecture of the driving system. A diagram explaining the representation of traffic flow at the lane level. A diagram explaining the representation of traffic flow at the grid level. A diagram explaining the representation of traffic flow at the segment level. A flowchart showing an example of how the driving system processes data. A diagram explaining an example of vehicle control. A diagram explaining an example of vehicle control. A diagram explaining an example of vehicle control. A diagram showing an example of the architecture of traffic flow recognition. A diagram explaining the combination of wide-area and narrow-area traffic flow. A diagram showing an example of the architecture of the driving system. A diagram showing an example of the architecture of the driving system. A diagram showing an example of the architecture of the driving system.
[0013] In the scope of this disclosure or claims, the term "processor" means one or more hardware processors configured to execute processing defined by computer program code (i.e., one or more instructions of a computer program) contained in a computer program, by reading the code each time. In other words, a "processor" is a hardware device that executes one or more programmed processes. Therefore, computer program code can also be considered software that can define the processing of the processor according to its content. For example, a "processor" may be a general-purpose or specific-purpose processor and may be, but is not limited to, a CPU, microprocessor, GPU, and DFP (Data Flow Processor).
[0014] In the scope of this disclosure or claims, the term “memory” means one or more hardware memories that are non-transitional tangible recording media configured to record computer program code and / or data in a manner accessible by a processor. “Memory” can be implemented by memory technologies such as SRAM, SDRAM, non-volatile flash memory, or other types of memory. The computer program code that constitutes the program is recorded in memory and executed by the processor, thereby enabling the processor to perform the various functions described above. “Memory” may further include volatile memory such as RAM (Random Access Memory).
[0015] In this disclosure or claims, the term “circuit” refers to one or more logic circuits as hardware, configured to perform specific processing based on a pre-designed circuit configuration. In other words (and, in contrast to “processor”), “circuit” in this disclosure or claims refers to hardware devices that perform specific processing based on a circuit configuration, rather than processing defined by software such as the computer program code described above. For example, “circuit” may include custom ICs such as ASICs (Application Specific Integrated Circuits) and FPGAs (Field Programmable Gate Arrays) designed with a Hardware Description Language (HDL). That is, “circuit” in this disclosure or claims includes all hardware circuits except for the processor described above, which performs processing by reading computer program code.
[0016] In the scope of this disclosure or claims, the expression “at least one circuit and processor” should be interpreted as a disjunctive (logical OR) and not as at least one circuit and at least one processor.
[0017] In this disclosure or claims, the term “processing unit” means a hardware device that performs processing by a “processor,” “circuit,” or a combination thereof. “Processing unit” may also mean the “processor” itself if its function is not achievable by the “circuit” but is achievable by the “processor.”
[0018] Several embodiments will be described below with reference to the drawings. In each embodiment, the same reference numerals are used for corresponding components, and redundant explanations may be omitted. If only a part of the configuration is described in each embodiment, the configuration of other embodiments described earlier can be applied to the other parts of that configuration. Furthermore, in addition to the combinations of configurations explicitly stated in the description of each embodiment, configurations from multiple embodiments can be partially combined even if not explicitly stated, as long as there are no particular problems with the combination.
[0019] (First Embodiment) The driving system DS of the first embodiment realizes functions related to the driving of the vehicle 1. The driving system DS may be the vehicle system itself, or it may be a component that constitutes a part of the vehicle system. Part or all of the driving system DS is installed in the vehicle 1. This vehicle 1 may be referred to as the own vehicle, host vehicle, etc.
[0020] Vehicle 1 may be a passenger car or a transport vehicle. Vehicle 1 can switch between operation at automation level 2 or higher, as defined in SAE J3016, and operation at level 0. Here, automation level 0 is no driving automation, and the driver performs all dynamic driving tasks (DDTs). Automation levels 1 and 2 are categorized as driving assistance. In particular, automation level 2 is partial driving automation, in which the system continuously performs longitudinal and lateral control of dynamic driving tasks within a specific Operational Design Domain (ODD). In this case, the driver is required to monitor the external environment and the system, etc.
[0021] Automation levels 3 to 5 are categorized as autonomous driving. Level 3 indicates that driving is conditionally automated. Level 4 indicates that driving is highly automated. Level 5 indicates that driving is fully automated. Here, "autonomous driving" in this embodiment means driving at automation level 2 or higher, including longitudinal and lateral driving control of dynamic driving tasks by the system.
[0022] The driving system DS combines recognition, judgment, and control functions to achieve autonomous driving of vehicle 1. Figure 1 shows an example of the hardware configuration of the driving system DS. For example, the driving system DS may include multiple electronic control units, such as a recognition ECU 10, a judgment ECU 11, and a control ECU 12.
[0023] The recognition ECU 10 is an electronic control unit configured primarily around a computer. The computer constituting the recognition ECU 10 has at least one processor 10a and at least one memory 10b. The processor 10a may be, for example, a CPU, but may also be of other types. The memory 10b may include a non-transitional physical storage medium that non-temporarily stores computer programs and data, such as ROM (Read Only Memory). The memory 10b may also include a rewritable volatile storage medium, such as RAM (Random Access Memory). RAM may be used for temporary storage of data being processed. The processor 10a is configured to be able to read the computer program stored in the memory 10b and can execute various processes according to the computer program.
[0024] The computer comprising the recognition ECU 10 may further have one or more AI units 11c for performing calculations using artificial intelligence (AI). The AI unit 11c operates when the processor 11a reads an AI control program stored in the memory 11b and is controlled by the processor 11a. The AI unit 11c has hardware optimized for processing artificial intelligence. The AI unit 11c may be implemented primarily using a GPU, primarily using an ASIC or FPGA, or a combination of these.
[0025] The decision ECU 11 is an electronic control unit mainly composed of a computer. The computer comprising the decision ECU 11 has at least one processor 11a and at least one memory 11b. The decision ECU 11 may further have an AI unit 11c. The processor 11a, memory 11b and AI unit 11c may have the same configuration as the processor 10a, memory 10b and AI unit 10c.
[0026] The control ECU 12 is an electronic control unit configured primarily with a computer. The computer constituting the control ECU 12 has at least one processor 12a and one memory 12b. The processor 12a and memory 12b may have the same configuration as the processors 10a, 11a and the memories 10b, 11b.
[0027] The driving system DS acquires a route from the navigation system 20 and also acquires data from recognition devices that detect the external environment, such as multiple external environment sensors 21, a map database (hereinafter referred to as map DB) 22, and a communication device 23. Based on the acquired route and data, the driving system DS performs processing and controls multiple motion actuators 24.
[0028] The navigation system 20 is installed in the vehicle 1 and is a system that presents and guides the driver or other occupant to the optimal route from the current location to the destination set by the occupant. The navigation system 20 uses an information display device 26 installed in the vehicle 1 to receive destination setting operations from the occupant and to present information to the occupant. The route set by the navigation system 20 includes specifying the roads to reach the destination, but does not have to include specifying lanes on multi-lane roads. The route may be directly specified by the occupant, or it may be selected by the occupant from multiple routes presented by the navigation system 20. Furthermore, the navigation system 20 does not have to be provided separately from the driving system DS, but may be incorporated into the driving system DS.
[0029] Multiple external environment sensors 21 are mounted on the vehicle 1 and detect targets present in the external environment of the vehicle 1. Examples of external environment sensors 21 include cameras, LiDAR (Light Detection and Ranging / Laser imaging Detection and Ranging), laser radar, millimeter-wave radar, ultrasonic sonar, and acoustic sensors. Typically, multiple types of external environment sensors 21 can be combined and implemented to monitor the front, side, and rear directions of the vehicle 1.
[0030] Furthermore, the driving system DS may include sensors other than the external environment sensor 21. The sensor data in this embodiment includes sensor data from the external environment sensor 21 and sensor data other than the external environment sensor 21. Sensors other than the external environment sensor 21 include sensors that detect the driving status of the vehicle 1 and sensors that detect the amount of manual operation by the driver. Sensors that detect the driving status of the vehicle 1 include, for example, a vehicle speed sensor, an acceleration sensor, and a yaw rate sensor. Sensors that detect the amount of manual operation by the driver include, for example, an accelerator sensor, a brake sensor, and a steering angle sensor. These sensor data may be used to handle relative speed and absolute speed, etc., in the traffic flow recognition described later.
[0031] Map DB 22 is a database installed in vehicle 1 that stores map data usable by the driving system DS. Map DB 22 is configured to include at least one type of non-transitory tangible storage medium, such as semiconductor memory, magnetic media, and optical media. Map DB 22 may include a database of PD maps generated using probe data (PD) collected from each vehicle traveling on public roads. Map DB 22 may include a database of high-precision maps with a high level of accuracy, mainly used for applications in automated driving systems. Map DB 22 may also include a database of parking maps, including detailed parking information such as parking space information, used for applications in automated parking or parking assistance. Map DB 22 may provide map data to the navigation system 20.
[0032] The communication device 23 is mounted on the vehicle 1 and acquires communication data usable in the driving system DS via wireless communication. The communication device 23 may also include a GNSS receiver that receives positioning signals from GNSS (global navigation satellite system) satellites present in the external environment of the vehicle 1.
[0033] Multiple motion actuators 24 are mounted on the vehicle 1 and can control vehicle motion based on control signals input from the driving system DS. The drive-type motion actuator 24 is a powertrain including at least one of the following: an internal combustion engine, a drive motor, etc. The braking-type motion actuator 24 is, for example, a brake actuator. The steering-type motion actuator 24 is, for example, a steering wheel.
[0034] Furthermore, the driving system DS may include an HCU (HMI Control Unit) 13 that comprehensively controls the HMI (Human Machine Interface) of the vehicle 1. The HCU 13 is an electronic control device mainly composed of a computer. The computer constituting the HCU 13 has at least one processor 13a and one memory 13b. The processor 13a and memory 13b may have the same configuration as the processors 10a, 11a, 12a and the memories 10b, 11b, 12b, but if a GPU is used as the processor 13a, it will be a configuration suitable for image display, which will be described later.
[0035] The HCU 13 acquires input signals from the operation input device 25. The HCU 13 also generates content for the information presentation device 26 to present information and controls the information presentation device 26.
[0036] The operation input device 25 is a device that receives input operations from a user such as a driver. The operation input device 25 may be a touch panel superimposed on a display, or it may be a mechanical steering switch located on the spokes of the steering wheel or the like.
[0037] The information presentation device 26 is a device that presents information to the user. The information presentation device 26 may be a display device that displays images, such as a meter display, a CID (center information display), or a HUD (head-up display). The information presentation device 26 may also be a sound-emitting device, such as a speaker or a buzzer. Information presentation may be carried out by combining the display device and the sound-emitting device.
[0038] Here, the functions of the driving system DS will be explained using Figure 2. The driving system DS includes a recognition unit F1 as a functional unit whose function is realized when the processor 10a executes a computer program stored in memory 10b. It also includes a planning unit F2 as a functional unit whose function is realized when the processor 11a executes a computer program stored in memory 11b. Furthermore, it includes an arbitration control unit F3 as a functional unit whose function is realized when the processors 11a and 12a execute computer programs stored in memories 11b and 12b. Finally, it includes a speed limit determination unit F4 as a functional unit whose function is realized when the processor 13a executes a computer program stored in memory 13b.
[0039] Here, the aforementioned functional units may include functions implemented using neural networks such as AI units 10c, 11c, etc. Such neural network-based processing will be referred to as AI-based processing below. On the other hand, processing that does not use neural networks and follows rules written in computer program code or rules obtained from a database, etc., will be referred to as rule-based processing below.
[0040] A neural network is an algorithm modeled after the human brain and, as mentioned above, is also called artificial intelligence. Neural networks primarily consist of multi-layer perceptrons. Each layer of a neural network is composed of multiple nodes called artificial neurons, and these neurons are connected to other neurons in the preceding and succeeding layers. The strength of these connections is called a weight.
[0041] A neural network may include, for example, pooling layers and fully connected layers, in addition to convolutional layers. Each convolutional layer performs an operation by convolving a local filter (called a kernel or feature extractor) of a predetermined size, defined for each convolutional layer, onto the input data (features). For example, the input value entered at the position corresponding to the neuron in the kernel is multiplied by a weight pre-set in the kernel for each neuron, and a linear sum of these multiplicative values is calculated. The convolution operation is performed by adding a bias pre-set in the kernel to this linear sum. The result of such a convolution operation may be transformed using, for example, a ReLU (Rectified Linear Unit) activation function. Furthermore, convolutional layers can be multilayered to perform the convolution operation and transformation by the activation function multiple times.
[0042] The pooling layer calculates local statistics of the feature map output from the convolutional layer, which is a higher layer connected to the input. Specifically, it sets a window of a predetermined size corresponding to the position of the higher layer and calculates local statistics from the input values within the window. For example, the maximum value may be used as the local statistic. That is, by adopting the maximum value of the input values within the window as the result of pooling, the input values within the window can be integrated.
[0043] A fully connected layer is a layer in which each neuron is connected to all neurons in the upper layer connected to the input side, and is mainly used for feature amount extraction, vector expression conversion, and the like. While a fully connected layer is relatively easy to implement, there also exists a problem that the number of parameters tends to be large, which easily increases the amount of arithmetic processing.
[0044] The recognition unit F1 recognizes the external environment of the vehicle 1 (in particular, recognizes targets) based on data acquired from the recognition device. The recognition unit F1 may be implemented on an AI basis or on a rule basis. Specifically, the recognition unit F1 includes a fusion unit F11 and a traffic flow recognition unit F12.
[0045] The fusion unit F11 fuses the sensor data acquired from the plurality of external environment sensors 21, the map data acquired from the map DB 22, and the position of the vehicle 1 estimated based on the positioning signal acquired from the communication device 23. The fusion unit F11 may generate an environment model by fusing these pieces of data. The environment model may be a model in which targets such as lanes, other agents, free spaces, and traffic signals are arranged in a virtual two-dimensional space or three-dimensional space with the vehicle 1 as a reference, based on map data and sensor data. The fusion unit F11 may further acquire route information from the navigation system 20, and may reflect the route information in the environment model. The fusion unit F11 sequentially updates the environment model based on the latest information.
[0046] The traffic flow recognition unit F12 recognizes the traffic flow of the road on which the vehicle 1 is traveling, using data indicating the external environment. The traffic flow recognition unit F12 may recognize the traffic flow using the environment model generated by the fusion unit F11, or may recognize the traffic flow based on sensor data, map data, and the like separately from the fusion unit F11. The traffic flow is a concept that captures the speed and direction of agents such as other vehicles traveling around the vehicle 1 as a flow. The traffic flow may be defined based on a plurality of agents. On the other hand, even when only one agent exists around the vehicle 1, the traffic flow may be defined based on the speed and direction of the single agent, so that processing using the concept of traffic flow can be employed.
[0047] In the present embodiment, that the vehicle 1 matches the traffic flow may mean that operations for surrounding vehicles to avoid collision with the vehicle 1 are kept to a minimum. For example, that the vehicle 1 matches the traffic flow may mean that the vehicle 1 travels at a speed substantially matching the speed of the traffic flow. The speed that substantially matches herein includes a speed having a difference of 10 km / h or less from the speed of the traffic flow. That the vehicle 1 disturbs the traffic flow may mean that the vehicle 1 travels at a speed having a difference exceeding 10 km / h from the speed of the traffic flow.
[0048] The traffic flow data (hereinafter referred to as traffic flow data) recognized by the traffic flow recognition unit F12 includes the speed of the traffic flow. The traffic flow data may include additional information such as the acceleration of the traffic flow, the direction of the traffic flow, changes in the direction of the traffic flow, and statistics related to speed, acceleration and direction (for example, mean, median, variance, standard deviation, distribution information).
[0049] The traffic flow output by the traffic flow recognition unit F12 as a recognition result can be expressed in various representation methods. As shown in FIG. 3, the traffic flow can be expressed by a lane-level representation method that indicates the traffic flow divided in units of lanes L1, L2, and L3 on the road where the vehicle 1 travels. For example, it is assumed that the three-lane road in FIG. 3 has a legal speed limit of 90 km / h.
[0050] Lane L1, the leftmost lane of a three-lane road, includes another vehicle OV1 traveling at 80 km / h in front of vehicle 1, and another vehicle OV2 traveling at 80 km / h behind vehicle 1. In one example of a traffic flow speed recognition method, other vehicles that are predicted to potentially interact with vehicle 1 are used as references for speed in the traffic flow, while other vehicles that are not predicted to potentially interact are excluded from the speed reference. The prediction of the possibility of interaction with vehicle 1 may include the possibility of other vehicles changing lanes. For example, the speed of other vehicle OV1, which is ahead of vehicle 1, is excluded without consideration, and the speed of other vehicle OV2, which is behind vehicle 1 and within the range recognizable by the external environment sensor 21, is referenced. Since the speed of other vehicle OV2 is 80 km / h, the speed of the traffic flow in lane L1 is 80 km / h. Furthermore, if there are multiple other vehicles behind vehicle 1 and within the range recognizable by the external environment sensor 21, the speed of the traffic flow may be the highest speed among the speeds of the multiple other vehicles, the average speed, or the median.
[0051] Lane L2, located in the center of a three-lane road, is the lane in which only vehicle 1 is traveling. The speed of vehicle 1 is excluded from the recognition of traffic flow. Therefore, lane L2 is considered a lane in which no target vehicles exist in the recognition of traffic flow. In lanes in which no target vehicles exist in the recognition of traffic flow, the speed of the traffic flow may be considered equal to the speed limit. Therefore, the speed of the traffic flow in lane L2 is 90 km / h.
[0052] Lane L3, located in the center of the three-lane road, includes other vehicle OV3 traveling at 120 km / h in front of vehicle 1, and other vehicle OV4 traveling at 100 km / h behind vehicle 1. As with lane L1, other vehicle OV3 is excluded, so the traffic flow speed in lane L3 is 100 km / h.
[0053] Furthermore, if there is a vehicle attempting to change lanes from lane L3 to lane L2, such as vehicle OV3, the direction of vehicle OV3 may be reflected in the direction of the traffic flow. If the direction of the traffic flow in lane L3 does not coincide with the direction of extension of lane L3, it may be inferred in subsequent processing such as in the planning section F2 that an obstacle has occurred in lane L3 that would necessitate a lane change.
[0054] As shown in Figure 4, traffic flow may be represented using a grid-level representation, where each lane L1, L2, L3 on the road on which vehicle 1 travels is further divided into grid units of predetermined length. In the grid-level representation, the speed of traffic flow in each grid may substantially coincide with the speed of other vehicles in that grid. If multiple other vehicles are in the same grid, the speed of traffic flow may be the highest speed among the multiple other vehicles, the average speed, or the median. In grids where no other vehicles are present, the speed of traffic flow may be considered equal to the speed limit, or it may not be defined at all.
[0055] As shown in Figure 5, traffic flow may be represented using a segment-level representation, where traffic flow is divided into segments based on vehicle 1. For example, a segment may be divided into segments corresponding to the front area of vehicle 1, segments corresponding to the lateral area, and segments corresponding to the rear area. The segment may be further divided into individual lanes L1, L2, and L3.
[0056] If multiple other vehicles are in the same grid, the speed of the traffic flow may be the highest speed among the multiple other vehicles, the average speed, or the median. In grids where no other vehicles are present, the speed of the traffic flow may be considered equal to the speed limit, or it may not be defined.
[0057] When traffic flow is represented using a grid-level or segment-level representation, depending on the purpose of using the traffic flow, post-processing may include integration processing in which multiple grids or segments are merged, or statistical processing in which each grid or segment is treated statistically.
[0058] The traffic flow recognized by the traffic flow recognition unit F12 may be recognized using a trajectory generation model mainly based on a neural network. When the AI unit 10c operates based on the loading of a computer program, the traffic flow recognition model is defined by the description of the neural network in the program code. Then, the neural network calculations are executed when a processor such as a GPU provided in the AI unit 10c loads the computer program. When the AI unit 10c operates in a form that does not require the loading of a computer program, the model is defined based on the circuit configuration such as an FPGA provided in the AI unit 10c, and the neural network calculations are executed by the operation of the circuit. Furthermore, these operating modes may be combined to execute the neural network calculations.
[0059] The traffic flow recognition model may receive an environmental model, or sensor data and map data as input. The traffic flow recognition model may also receive commands specifying the representation method of the traffic flow, commands specifying the accuracy of the traffic flow, etc. In response to the input, the traffic flow recognition model outputs traffic flow data on the roads surrounding vehicle 1 using one of the above-mentioned representation methods.
[0060] The planning unit F2 formulates a plan for autonomously driving vehicle 1 based on information about the external environment recognized by the recognition unit F1. Here, the information about the external environment includes traffic flow data recognized by the traffic flow recognition unit F12. The planning unit F2 includes a behavior planning unit F21 and a trajectory planning unit F22. The behavior planning unit F21 may be implemented on an AI basis or on a rule basis. The trajectory planning unit F22 may also be implemented on an AI basis or on a rule basis, but the following explanation will continue assuming that the trajectory planning unit F22 is implemented on an AI basis.
[0061] The behavior planning unit F21 plans the behavior of vehicle 1 based on route information obtained from the navigation system 20 and the latest environmental model obtained from the fusion unit F11. Specifically, if there are multiple lanes on the set route, the behavior planning unit F21 may plan the lane on which vehicle 1 will travel and the lane change plan, referring to the direction to the destination.
[0062] The behavior planning unit F21 also determines the basic behavior of vehicle 1. For example, if vehicle 1 is moving, the behavior planning unit F21 may refer to the status of traffic lights and the presence or absence of obstacles in front of vehicle 1 and plan whether to continue driving or stop. If vehicle 1 is stopped, the behavior planning unit F21 may plan whether to continue stopping or start moving. The behavior planning unit F21 may also refer to route information and plan the direction of vehicle 1, such as going straight, turning right, or turning left.
[0063] The behavior planning unit F21 may refer to the execution conditions for MRM (Minimal Risk Maneuver) and plan the execution of MRM. MRM is the behavior of vehicle 1 to transition vehicle 1 to a Minimal Risk Condition (MRC), and may also be called an emergency stop. The plan planned by the behavior planning unit F21 is provided to the trajectory planning unit F22.
[0064] The trajectory planning unit F22 plans the trajectory of vehicle 1 based on the latest environmental model obtained from the fusion unit F11 and the behavior plan planned by the behavior planning unit F21. Trajectory planning is performed repeatedly at predetermined time intervals. Trajectory planning includes so-called path planning, which plans the specific coordinates and orientation (yaw) of vehicle 1 as it actually travels along the path, and also includes a speed plan for when it travels along the path.
[0065] The trajectory planning unit F22 predicts the behavior or trajectory of each agent, such as other vehicles, recognized around vehicle 1, based on the environmental model. Based on the prediction results for each agent, the trajectory planning unit F22 generates multiple trajectories (hereinafter also referred to as planned trajectories) for vehicle 1 to travel. The trajectory planning unit F22 in this embodiment is capable of generating multiple types of trajectories, including one or more traffic flow-aligned trajectories T1 and one or more set-aligned trajectories T2.
[0066] The traffic flow alignment trajectory T1 is a trajectory that aligns with the traffic flow recognized by the traffic flow recognition unit F12. A trajectory that aligns with the traffic flow can also be described as a trajectory that restricts vehicle 1 from disrupting the traffic flow. A trajectory that aligns with the traffic flow or a trajectory that restricts disruption of the traffic flow may also be a trajectory that reduces the number and amount of operations required for each agent to respond to vehicle 1. The operations required for responding to vehicle 1 here are an example of an interaction with vehicle 1. A trajectory that aligns with the traffic flow or a trajectory that restricts disruption of the traffic flow may also be a trajectory that reduces the number and amount of operations required for each agent to respond to vehicle 1. A trajectory that aligns with the traffic flow or a trajectory that restricts disruption of the traffic flow may also be a trajectory that reduces the possibility of vehicle 1 causing traffic congestion on the road it is traveling on.
[0067] For example, consider a scenario where the traffic flow speed is 80 km / h, and vehicle 1 travels at 40 km / h according to its planned trajectory. In this case, other vehicles following vehicle 1 in the lane would need to either significantly reduce their speed to 40 km / h or change lanes to keep up with vehicle 1. Such a planned trajectory forces other vehicles to take action to keep up with vehicle 1, and therefore cannot be considered a trajectory that aligns with the traffic flow.
[0068] The configured trajectory T2 is a trajectory that is consistent with the pre-configured settings (hereinafter referred to as the autonomous driving settings), regardless of traffic flow. The autonomous driving settings may be settings based on a driving policy that is pre-defined by default in the driving system DS. For example, if the driving policy assumes that vehicle 1 will travel at the legal speed limit, then the configured trajectory T2 may be a trajectory that effectively drives vehicle 1 at the speed limit.
[0069] Furthermore, the autonomous driving settings may be configured by the user operating the control input device 25. The autonomous driving settings may include, for example, setting an absolute value for the upper speed limit. In this case, the user can set any value, such as 40 km / h, as the upper speed limit. The autonomous driving settings may also include setting an upper speed limit relative to the speed limit imposed on each road. The user can set speeds such as speed limit minus 10 km / h, speed limit plus 10 km / h, etc., as the upper speed limit.
[0070] The autonomous driving settings may include setting a target speed, either in place of or in conjunction with setting an upper speed limit. For example, if the user sets the vehicle 1 to travel at a target speed such as 40 km / h, the set-aligned trajectory T2 may be a trajectory that substantially drives the vehicle 1 at 40 km / h. It is also permissible for the speed to momentarily exceed 40 km / h, such as on a downhill slope.
[0071] The planned trajectory planned by the trajectory planning unit F22 can be generated using a trajectory generation model primarily based on a neural network. When the AI unit 11c operates based on the loading of a computer program, the trajectory generation model is defined by the description of the neural network in the program code. Then, the neural network calculations are executed when a processor such as a GPU provided in the AI unit 11c loads the computer program. When the AI unit 11c operates in a form that does not require the loading of a computer program, the trajectory generation model is defined based on the circuit configuration such as an FPGA provided in the AI unit 11c, and the neural network calculations are executed by the operation of the circuit. Furthermore, these operating modes may be combined to execute the neural network calculations.
[0072] The trajectory generation model receives input such as an environment model or information shown in the environment model, information related to behavior planning, user-defined autonomous driving settings, and traffic flow data. The trajectory generation model may also receive commands to generate one or more traffic flow-aligned trajectories T1 and one or more set-aligned trajectories T2, or it may be configured so that, through prior training of the trajectory generation model, one or more traffic flow-aligned trajectories T1 and one or more set-aligned trajectories T2 are generated without requiring such commands. The trajectory generation model may also receive commands specifying the trajectory inference time.
[0073] In response to the input, the trajectory generation model outputs one or more trajectory data sets, each containing one or more traffic flow-aligned trajectories T1 and one or more set-aligned trajectories T2. The trajectory data stores one or more sets of a trajectory planned for vehicle 1 and the confidence level (which can also be called probability) of that trajectory.
[0074] Each trajectory includes a path representing the coordinates (x, y) that vehicle 1 passes through in time series, a yaw angle (θ) indicating the direction of vehicle 1 associated with each coordinate constituting the path, and the velocity of vehicle 1 associated with each coordinate constituting the path. Furthermore, each trajectory may also include the acceleration of vehicle 1 associated with each coordinate constituting the path. The trajectory is output based on the inferred trajectory time length. For example, if the inferred trajectory time length is 5 seconds, the trajectory of vehicle 1 from the planning time to 5 seconds later may be output.
[0075] The reliability of the trajectory represents its recommendation level in the planning of Vehicle 1. On the other hand, the actual trajectory that Vehicle 1 travels may be determined by rules, with other factors such as the user's intentions taking into account. Therefore, the trajectory with the highest reliability is not necessarily selected as the trajectory used to control Vehicle 1.
[0076] Furthermore, the trajectory generation model may be configured to output each trajectory with a type label attached so that the type of output trajectory (traffic flow-aligned trajectory T1 or set-aligned trajectory T2) can be determined. On the other hand, the trajectory generation model may not attach a type label at the time of output, and the planning unit F2 or control unit F32 may analyze each trajectory in a rule-based manner to determine the type of trajectory.
[0077] The arbitration control unit F3 determines the trajectory to be actually adopted based on the planned trajectory obtained from the planning unit F2, and controls each motion actuator 24. The arbitration control unit F3 is implemented on a rule basis. The arbitration control unit F3 includes an arbitration unit F31 and a control unit F32. Of these, the arbitration unit F31 may be implemented by the decision ECU 11, and the control unit F32 may be implemented by the control ECU 12.
[0078] The mediation unit F31 mediates between multiple planned trajectories obtained from the planning unit F2 to derive a single final planned trajectory. For example, if the planning unit F2 provides a traffic flow alignment trajectory T1 and a set alignment trajectory T2, the mediation unit F31 determines which trajectory to adopt. The mediation unit F31 may also determine the trajectory to adopt (hereinafter also referred to as the mediated trajectory) based on information about the user's intentions provided by the speeding judgment unit F4.
[0079] The control unit F32 refers to the motion performance of the vehicle 1 and determines whether it is possible to drive the vehicle 1 based on the arbitration trajectory arbitrated by the arbitration unit F31. If it is possible, the control unit F32 converts the arbitration trajectory into control signals for each motion actuator 24 and outputs the control signals to each motion actuator.
[0080] The control unit F32 rejects the control of vehicle 1 based on an arbitration trajectory that does not allow vehicle 1 to travel in terms of its motion performance. If the latest arbitration trajectory is rejected, the control unit F32 does not update the trajectory used for control, but instead continues to drive vehicle 1 according to the trajectory that was output in the previous cycle and is currently being used for control.
[0081] The arbitration unit F31 may store the trajectory actually used for control and the decision result that led to the adoption of that trajectory from among multiple planned trajectories in a corresponding manner in a storage medium such as memory 11b, 12b. This data may be transmitted to a server that performs data collection via the communication device 23 or through a dealer, workshop intermediary, etc. The data collected by the server may be used to improve the traffic flow recognition model or trajectory generation model, improve the algorithms of the arbitration unit F31 or control unit F32, etc.
[0082] The speeding violation determination unit F4 makes a determination regarding whether speeding is permissible. Permissible speeding, as used here, may mean allowing the vehicle 1 to travel in a trajectory that exceeds the speed limit. The speeding violation determination unit F4 is implemented on a rule basis. The speeding violation determination unit F4 includes a case-by-case determination unit F41 and an experience-based determination unit F42.
[0083] The decision-making unit F41 confirms the user's intentions each time and makes a decision on whether to allow speeding based on these confirmations. The decision-making unit F41 confirms the user's intentions using the operation input device 25 and the information presentation device 26. Specifically, the decision-making unit F41 causes the information presentation device 26 to present to the user content requesting the user to indicate whether or not they will allow speeding, and content for the user to respond.
[0084] The judgment unit F41 checks the user's input result to the operation input device 25 in response to the presented content, and based on the input result, determines whether the current user's intention is to permit speeding or to prohibit speeding.
[0085] The decision unit F41 provides the mediation unit F31 with information regarding the user's intention, including the result of this decision. If the user's intention indicates that speeding is permissible, the mediation unit F31 permits the adoption of a trajectory that exceeds the speed limit. However, based on evaluations of other decision factors, the mediation unit F31 may ultimately adopt a trajectory that causes vehicle 1 to travel at or below the speed limit. If the user's intention indicates that speeding is prohibited, the mediation unit F31 prohibits the adoption of a trajectory that exceeds the speed limit. For example, if the traffic flow aligning trajectory T1 is a trajectory that causes vehicle 1 to travel at or above the speed limit, and the set aligning trajectory T2 is a trajectory that causes vehicle 1 to travel at substantially the speed limit, the mediation unit F31 prohibits the adoption of the traffic flow aligning trajectory T1 and adopts the set aligning trajectory T2.
[0086] Furthermore, the decision unit F41 stores information regarding the user's intentions in the experience database (hereinafter referred to as the experience DB) 13c. The experience DB 13c may be provided by the HCU 13, the decision ECU 11, or the control ECU 12. The experience DB 13c may also be provided independently outside the ECUs, etc., in the driving system DS. The experience DB 13c is configured to include at least one type of non-transitional physical storage medium, such as a semiconductor memory, a magnetic medium, and an optical medium.
[0087] The information regarding the user's intentions stored in the experience DB13c here includes situational information representing the scene or scenario that vehicle 1 is encountering when the user expresses their intention, and the result of the user's judgment at that time. In this way, information regarding the user's intentions is accumulated for each scene or scenario.
[0088] Furthermore, the decision unit F41 does not need to present content each time to confirm the user's intentions. For example, the decision unit F41 may check the user's prior autonomous driving settings, and if those settings indicate that speeding is permitted, it may determine that the user currently intends to allow speeding. For example, if the autonomous driving settings have set a speed limit of the speed limit plus 10 km / h as the upper speed limit, it can be considered that the user intends to allow speeding.
[0089] The experience judgment unit F42 provides a function to determine whether speeding is permissible based on information about past user intentions stored in the experience DB13c, in order to avoid having to confirm the user's intentions each time and to reduce the burden on the user. This function may be enabled and disabled in the same way as the autonomous driving settings.
[0090] Specifically, if its function is enabled, the experience judgment unit F42 first refers to the recognition result or environmental model to identify the scene or scenario that vehicle 1 is currently encountering. Then, the experience judgment unit F42 refers to the experience DB 13c to identify what decisions the user has made in the past in scenes or scenarios that match or are similar to the identified scene or scenario. Based on the identified decision result, the experience judgment unit F42 decides whether to allow or prohibit speeding. If there are no past decision results for the relevant scene or scenario, or if there is a large variation in the decision results, the experience judgment unit F42 may decide not to automatically decide whether to allow speeding in the driving system DS, but to confirm the user's intention. If it is decided to confirm the user's intention, the judgment unit F41 makes the decision as described above each time.
[0091] Next, an example of a processing method implemented by the operating system DS will be explained using several steps S101 to S110 of the flowchart in Figure 6.
[0092] In the first step, S101, the recognition unit F1 recognizes the external environment. In S102, following S101, the recognition unit F1 recognizes the traffic flow based on the external environment and estimates the speed of the traffic flow. In S103, following S102, the planning unit F2 generates a planned trajectory, and the mediation control unit F3 obtains candidate planned trajectories.
[0093] In S104, following S103, the arbitration control unit F3 determines whether there is a trajectory that exceeds the speed limit among the planned trajectories acquired in S103. If yes, proceed to S105. If no, proceed to S109.
[0094] In S105, the speed limit determination unit F4 determines whether speed limiting is permitted in the pre-configured autonomous driving settings. If yes, proceed to S108. If no, proceed to S106.
[0095] In S106, the speeding judgment unit F4 refers to the experience DB13c and determines whether the user has tolerated speeding in similar past situations. If yes, proceed to S108. If no, proceed to S107.
[0096] In S107, the speed limit determination unit F4 uses the information presentation device 26 to present content to the user requesting them to indicate whether or not they will tolerate speeding, and content for their response. After the user responds using the operation input device 25, the speed limit determination unit F4 determines whether or not the user currently tolerates speeding. If the answer is Yes, the process proceeds to S108. If the answer is No, the process proceeds to S109.
[0097] In S108, the mediation control unit F3 performs mediation considering the traffic flow, assuming that exceeding the speed limit is permitted. On the other hand, in S109, the mediation control unit F3 performs mediation, assuming that exceeding the speed limit is prohibited. After the mediation trajectory is determined in S108 or S109, the process proceeds to S110.
[0098] In S110, the arbitration control unit F3 controls the movement of vehicle 1 based on the arbitration trajectory. The series of processes ends with S110.
[0099] Next, using Figures 7 to 9, an example of vehicle 1 control by the driving system DS in a specific scenario will be explained. In the scenario in Figure 7, one of the three lanes on a three-lane road is restricted due to construction. As a result of this lane restriction, a sign RS1 with a speed limit of 20 km / h is installed before the lane restriction point PRL. Consequently, the speed limit is 60 km / h in the first section A1 before sign RS1, while it becomes 20 km / h in the second section A2 after sign RS1.
[0100] On the other hand, most of the other vehicles OV and OV5 do not take any deceleration action before entering the second section A2, but after entering the second section A2 they decelerate from about 60 km / h to about 40 km / h and pass through the second section A2 at 40 km / h. At this time, the traffic flow recognition unit F12 recognizes a traffic flow of 60 km / h in the first section A1, recognizes 60 km / h at the starting point of the second section A2, and recognizes a traffic flow of 40 km / h throughout the entire area of the second section A2 except for the starting point.
[0101] The trajectory planning unit F22 generates both the traffic flow-aligned trajectory T1 and the set-aligned trajectory T2. For example, the traffic flow-aligned trajectory T1 is a trajectory in which vehicle 1 does not decelerate before entering the second section A2, decelerates from approximately 60 km / h to approximately 40 km / h after entering the second section A2, and passes through the second section A2 at 40 km / h. This trajectory is one in which vehicle 1 exceeds the speed limit in the second section A2.
[0102] On the other hand, if the autonomous driving setting is set to use the speed limit as the upper speed limit, the set-aligned trajectory T2 is a trajectory in which the vehicle decelerates from 60 km / h to 20 km / h by the end of the first section A1, and then travels at a constant speed of 20 km / h through the second section A2. In this trajectory, the vehicle decelerates sharply at the end of the first section A1, leaving a remaining risk that vehicle 1 will be rear-ended by other vehicles OV and OV5.
[0103] On the other hand, if the on-demand decision unit F41 recognizes lane closures and speed limit trajectories due to construction based on map data, etc., it confirms the user's intention regarding speeding while vehicle 1 is traveling through the first section A1. If it is determined that the time until vehicle 1 enters the second section A2 is shorter than the time required to confirm the user's intention, the speeding determination by the experience decision unit F42 may be adopted instead of the user's intention confirmation by the on-demand decision unit F41.
[0104] Then, based on the judgment information regarding the permissibility of speeding obtained by the speeding judgment unit F4, the mediation unit F31 adopts the traffic flow alignment trajectory T1 if speeding is permitted. If speeding is prohibited, the mediation unit F31 adopts the set alignment trajectory T2. The vehicle 1 is then controlled based on the trajectory thus adopted.
[0105] Furthermore, the driving system DS may be configured to allow temporary speed overruns at points where the speed limit is changed in the autonomous driving settings. If such a setting is made, the set-aligned trajectory T2 may be a trajectory in which the vehicle temporarily exceeds the speed limit by decelerating from 60 km / h to 20 km / h after entering the second section A2, but then continues to travel at a constant speed of 20 km / h through the second section A2. When such a trajectory is adopted, the vehicle 1 travels in line with the traffic flow, which reduces the risk of the vehicle 1 being rear-ended by other vehicles OV, OV5, compared to a trajectory that strictly adheres to the speed limit.
[0106] In the scenario shown in Figure 8, vehicle 1 merges from merging lane L5 onto a two-lane road. The speed limit for merging lane L5 is 60 km / h, and the speed limit for the merging lane L4 is 100 km / h. Other vehicles OV, which are continuously traveling in the merging lane L4, are traveling at 100 km / h. At this time, the traffic flow recognition unit F12 recognizes a traffic flow of 100 km / h in lane L1.
[0107] The trajectory planning unit F22 generates both the traffic flow-aligned trajectory T1 and the set-aligned trajectory T2. For example, the traffic flow-aligned trajectory T1 is the trajectory in which vehicle 1 accelerates to 100 km / h in the merging lane L5 and changes lanes from the merging lane L5 to the merging lane L4 while at 100 km / h.
[0108] On the other hand, if the autonomous driving setting is set to use the speed limit as the upper speed limit, the setting-aligned trajectory T2 is a trajectory in which vehicle 1 travels at 60 km / h in the merging lane L5, changes lanes from the merging lane L5 to the merging lane L4 while at 60 km / h, and then accelerates vehicle 1 to 100 km / h. In this trajectory, when entering lane L4, following vehicles traveling in lane L4 will need to decelerate rapidly, which may cause congestion.
[0109] On the other hand, when the on-demand judgment unit F41 recognizes a merging point using map data or the like, it confirms the user's intention regarding speeding while the vehicle 1 is still in motion. If it is determined that the time to reach the merging point is shorter than the time required to confirm the user's intention, the speeding determination by the experience judgment unit F42 may be adopted instead of the user's intention confirmation by the on-demand judgment unit F41. The processing in the mediation unit F31 is the same as in the example in Figure 7.
[0110] The scenario in Figure 9 is one in which vehicle 1 is traveling in the left lane L6 of a two-lane road. The speed limit is 40 km / h throughout the entire section. The traffic flow recognition unit F12 recognizes a traffic flow of 55 km / h in the left lane L6 and a traffic flow of 60 km / h in the right lane L7. Here, vehicle 1 is traveling at the speed limit of 40 km / h, and another vehicle OV6 following behind is traveling in lane L6 at 55 km / h and is gradually closing the distance between itself and vehicle 1.
[0111] The trajectory planning unit F22 generates both a traffic flow-aligned trajectory T1 and a set-aligned trajectory T2. The traffic flow-aligned trajectory T1 is the trajectory that accelerates vehicle 1 from 40 km / h to 55 km / h, which is a speed that aligns with the traffic flow. On the other hand, if the autonomous driving setting is set to use the speed limit as the upper limit speed, the set-aligned trajectory T2 is the trajectory that drives vehicle 1 at 40 km / h.
[0112] The mediation unit F31 determines whether the distance between vehicle 1 and the following vehicle OV6 is greater than a preset threshold. The threshold used may be an experimentally obtained static optimal value, such as 30m. Instead of the distance between vehicles, the mediation unit F31 may determine whether the collision margin time with the other vehicle OV6 is greater than a preset threshold.
[0113] If the following distance or collision margin is greater than the threshold, the mediation unit F31 does not adopt the traffic flow matching trajectory T1 and adopts the set matching trajectory T2. If the following distance or collision margin is less than or equal to the threshold, the mediation unit F31 obtains judgment information regarding the permissibility of speeding, and adopts the traffic flow matching trajectory T1 if speeding is permitted. If speeding is prohibited, the mediation unit F31 adopts the set matching trajectory T2.
[0114] According to the first embodiment described above, the speed of the traffic flow on the road on which the vehicle 1 is traveling is determined based on sensor data, and a planned trajectory (trajectory aligned with traffic flow T1) is generated that allows the vehicle 1 to travel at a speed that matches the speed of the traffic flow. By adopting this generated planned trajectory, it becomes possible to align the behavior of the vehicle 1 with the traffic flow on the road, thereby suppressing the occurrence of traffic flow disturbances. As a result, the possibility of traffic disruptions can be reduced.
[0115] Furthermore, according to the first embodiment, traffic flow is calculated based on the speeds of agents present around vehicle 1, which are recognized based on sensor data. Since traffic flow is calculated based on the speeds of the actually recognized agents, and vehicle 1 is controlled based on this traffic flow, the error between the calculated traffic flow and reality can be reduced, thereby suppressing the occurrence of actual traffic flow disturbances. In addition, traffic flow is represented using a representation method that includes speeds expressed for each predetermined unit that divides the road area. Even in planned trajectories that include movement to other divided areas, it is possible to ensure alignment with the local traffic flow in those other areas, thereby further suppressing the occurrence of traffic flow disturbances.
[0116] Furthermore, according to the first embodiment, when generating a planned trajectory, a plurality of planned trajectories are generated, including a planned trajectory in which vehicle 1 travels at a speed matching the speed of the traffic flow (traffic flow-aligned trajectory T1), and a planned trajectory in which vehicle 1 travels at a speed different from the speed of the traffic flow. If there are concerns about adopting the traffic flow-aligned trajectory T1, it becomes possible to adopt a planned trajectory in which vehicle 1 travels at a speed different from the speed of the traffic flow, thus enabling flexible responses.
[0117] Furthermore, according to the first embodiment, the planned trajectory for driving vehicle 1 at a speed different from the speed of the traffic flow is a planned trajectory for driving vehicle 1 at a speed based on a speed setting that has been pre-set by the user of vehicle 1. By reflecting the user's settings, autonomous driving can be brought closer to the driving manner desired by the user.
[0118] Furthermore, according to the first embodiment, if the planned trajectory for driving vehicle 1 at a speed matched to the speed of the traffic flow exceeds the speed limit, the system determines whether or not to adopt the planned trajectory for driving vehicle 1 at a speed matched to the speed of the traffic flow as the trajectory used to control vehicle 1, based on information regarding the permissibility of exceeding the speed limit provided from outside the driving system DS. In other words, since the information regarding the permissibility of exceeding the speed limit used to select the trajectory is provided from an external source, it is possible to suppress the occurrence of traffic flow disturbances while ensuring the validity of the specifications of the driving system DS.
[0119] Furthermore, according to the first embodiment, the adoption or rejection of a generated planned trajectory that exceeds the speed limit is determined based on the user's confirmation of intent. For example, if the user indicates that exceeding the speed limit is acceptable, the adoption of a planned trajectory that exceeds the speed limit is permitted. Therefore, even if the speed of the traffic flow exceeds the speed limit, it becomes possible to control vehicle 1 in a way that minimizes disruption to the traffic flow. Thus, it becomes possible to reduce the possibility of traffic disruptions.
[0120] Furthermore, according to the first embodiment, a system is provided in which situational information representing the situation encountered by the vehicle 1 when the user of the vehicle 1 expresses an intention to exceed the speed limit, and an experience DB 13c that stores the result of the user's judgment at that time. The situation currently encountered by the vehicle 1 is identified. Furthermore, based on the experience DB 13c, it is identified what kind of intentions the user has expressed in the past in situations that match or are similar to the identified situation. If the user's past intentions indicate prohibition of exceeding the speed limit, the adoption of a planned trajectory that exceeds the speed limit is prohibited, and if the user's past intentions indicate permission to exceed the speed limit, the adoption of a planned trajectory that exceeds the speed limit is permitted. By making a judgment on whether to permit exceeding the speed limit based on past experience, the need to confirm the user's intention each time can be reduced. As a result, the inconvenience felt by the user can be suppressed.
[0121] Furthermore, according to the first embodiment, multiple planned trajectories are generated, and one trajectory is selected from the multiple planned trajectories to be used for controlling vehicle 1. In this configuration, at least one of the processes of determining the speed of traffic flow and generating planned trajectories is realized based on processing by an artificial intelligence model using a neural network. Trajectory selection is realized based on rule-based processing. In the black-box artificial intelligence model-based processing, it is difficult to completely eliminate the output of results that may cause traffic disruptions, or to prove that such disruptions are not possible. Therefore, while the output of results that may cause traffic disruptions in the artificial intelligence model-based processing is tolerated, rule-based processing is adopted in the trajectory selection as a post-processing step to exclude the adoption of such results. Thus, the validity of the specifications of the driving system DS can be improved.
[0122] (Second Embodiment) As shown in Figures 10 and 11, the second embodiment is a modification of the first embodiment. The second embodiment will be described focusing on the differences from the first embodiment.
[0123] The traffic flow recognition unit F12a of the second embodiment recognizes wide-area traffic flow (WTF) and narrow-area traffic flow (NTF). To this end, as shown in Figure 10, the traffic flow recognition unit F12a includes a communication recognition unit F12b, a sensor recognition unit F12c, and a traffic flow fusion unit F12d.
[0124] The communication recognition unit F12b recognizes the wide-area traffic flow (WTF) by sequentially downloading from the traffic flow server 90 via the communication device 23, or by periodically downloading from the traffic flow server 90 via the communication device 23 and storing it in the map DB 22.
[0125] Wide-area traffic flow (WTF) can be represented, for example, at the road node level. A road node is a road section defined to represent the connection relationships of roads, with boundaries such as points where roads branch or merge, or the ends of roads. If the section length of a road node is long, the traffic flow may be represented for each section obtained by dividing the node up to a predetermined length (e.g., 200m, 300m), as shown in Figure 11. In this way, wide-area traffic flow (WTF) includes the road on which vehicle 1 travels and represents traffic flow in an area wider than the detection area of the vehicle 1's external environment sensor 21. Furthermore, since wide-area traffic flow (WTF) represents traffic flow with lower resolution than narrow-area traffic flow (NTF), it may also be called low-resolution traffic flow.
[0126] The traffic flow server 90 measures wide-area traffic flow WTF by, for example, collecting vehicle speed data, GNSS positioning information, or self-position estimation results from each vehicle traveling on the road via V2I communication. Alternatively, the traffic flow server 90 may measure wide-area traffic flow WTF by collecting road imaging results from a roadside unit equipped with an infrastructure camera that images the road.
[0127] The sensor recognition unit F12c recognizes the local traffic flow (NTF) primarily using sensor data from the external environmental sensor 21. The sensor recognition unit F12c may also refer to map data obtained from the map DB 22 in order to recognize traffic flow using road-based representation methods at the lane level, grid level, or segment level. The local traffic flow (NTF) may be the same as the traffic flow recognized in the first embodiment. That is, the local traffic flow (NTF) indicates traffic flow in a narrow area around the vehicle 1, equivalent to the detection area of the external environmental sensor 21. Since the local traffic flow (NTF) represents traffic flow with higher resolution than the local traffic flow (WTF), it may also be called high-resolution traffic flow.
[0128] The traffic flow fusion unit F12d fuses the wide-area traffic flow (WTF) recognized by the communication recognition unit F12b with the narrow-area traffic flow (NTF) recognized by the sensor recognition unit F12c. The fusion here may involve combining the two types of traffic flow data into a single traffic flow data with partially different resolutions. Alternatively, the fusion here may involve combining the two types of traffic flow data into a single traffic flow data with a two-tiered data container.
[0129] The trajectory planning unit F22, which receives traffic flow data in which such wide-area traffic flow WTF and narrow-area traffic flow NTF are fused, can generate a traffic flow-aligned trajectory T1 that is consistent with the wide-area traffic flow WTF, even outside the area covered by the narrow-area traffic flow NTF.
[0130] According to the second embodiment described above, the traffic flow includes both a narrow-area traffic flow (NTF) that represents the traffic flow in a narrow area around the vehicle 1 recognized based on sensor data with high resolution, and a wide-area traffic flow (WTF) distributed from the traffic flow server 90 that represents the traffic flow in a wider area than the narrow-area traffic flow (NTF) with lower resolution than the narrow-area traffic flow. By combining traffic flow information with such different characteristics, the consistency of the planned trajectory with the traffic flow can be further improved.
[0131] (Third Embodiment) As shown in Figure 12, the third embodiment is a modification of the first embodiment. The third embodiment will be described focusing on the differences from the first embodiment.
[0132] In the third embodiment of the driving system DS, the recognition unit F101 is implemented on an AI basis and processes recognition of the external environment, including the recognition of traffic flow, using a recognition model as an integrated neural network.
[0133] The recognition model may receive route information, sensor data, and map data from the navigation system 20. The recognition model may also receive commands specifying the method of representing traffic flow, commands specifying the accuracy of traffic flow, and so on.
[0134] In response to the input, the recognition model outputs object recognition data indicating the current position and state of objects around vehicle 1, and traffic flow data. These objects include signs displaying speed limits. The object recognition data may be output in a form separated for each object, or in the form of an environmental model. The traffic flow data may be output in a form superimposed on the environmental model, based on the representation methods shown in the first and second embodiments.
[0135] The output data from this recognition model is provided to the planning unit F201. In this third embodiment, the planning unit F201 is implemented on an AI basis and processes behavior planning and trajectory planning using a planning model as an integrated neural network.
[0136] The planning model receives object recognition data, traffic flow data, and information on autonomous driving settings. In response to these inputs, the trajectory generation model outputs one or more trajectory data sets, each containing one or more traffic flow-aligned trajectories T1 and one or more setting-aligned trajectories T2.
[0137] As in the third embodiment, when data is passed from one neural network to another, at least a portion of the data may be in the form of high-dimensional data for inter-neural network input and output. For example, traffic flow data may be in the form of high-dimensional data. High-dimensional data is data containing a large number of features and may be decipherable or undecipherable by humans. Human-decipherable traffic flow data may be represented in image format and visualized in map format, for example. Human-undecipherable traffic flow data may be in the form of a feature map calculated by a recognition model.
[0138] Furthermore, the recognition model may provide traffic flow data to the mediation control unit F3. Since the mediation control unit F3 will use the traffic flow data for rule-based processing, the traffic flow data provided to the mediation control unit F3 does not need to be high-dimensional data.
[0139] According to the third embodiment described above, the understanding of traffic flow speed is achieved based on the processing of the first model (recognition model) as an artificial intelligence model. The generation of the planned trajectory is achieved based on the processing of the second model (planning model) as an artificial intelligence model. The traffic flow data representing traffic flow, output from the first model and input to the second model, is passed between the first and second models in the form of high-dimensional data containing a large number of features. By adopting high-dimensional data, the data exchange between neural networks based on feature calculations is made more efficient. Since it is possible to generate a planned trajectory while suppressing the occurrence of processing delays, the latest traffic flow speed can be accurately reflected in the autonomous driving of vehicle 1.
[0140] (Other Embodiments) Although several embodiments have been described above, this disclosure is not to be construed as being limited to those embodiments, and can be applied to various embodiments and combinations without departing from the spirit of this disclosure.
[0141] As another embodiment relating to the first and third embodiments, as shown in Figure 13, the traffic flow recognition unit F1y may be provided independently of the recognition unit F1x. In this case, the traffic flow recognition unit F1y may be implemented by a traffic flow recognition-dedicated ECU separate from the recognition ECU 10.
[0142] Furthermore, in another embodiment, as shown in Figure 14, the driving system DS may have a recognition and planning unit F102 that integrates the recognition unit F1 and the planning unit F2, instead of having a recognition unit F1 and a planning unit F2, respectively. The recognition and planning unit F102 is implemented on an AI basis and processes recognition functions, including the recognition of traffic flow, and planning functions using a recognition and planning model as an integrated neural network.
[0143] The recognition planning model may receive route information, sensor data, and map data from the navigation system 20. The recognition model may also receive commands specifying the method of representing traffic flow, commands specifying the accuracy of traffic flow, and so on.
[0144] In response to such inputs, the recognition planning model can infer the location and state of targets, traffic flow, the trajectory of vehicle 1, and the behavior and intentions of other agents, such as other vehicles, present around vehicle 1.
[0145] For example, a recognition planning model outputs multiple types of trajectories, including one or more traffic flow-aligned trajectories T1 and one or more pre-configured trajectories T2, along with traffic flow data. The trajectories may be output together with a confidence level.
[0146] Furthermore, in other embodiments, the hardware configuration of the driving system DS may be modified as appropriate. For example, in the arbitration control unit F3, the arbitration unit F31 may be implemented by the judgment ECU 11, and the control unit F32 may be implemented by the control ECU 12. Also, for example, at least some of the functions of the speeding judgment unit F4 may be implemented by one or both of the judgment ECU 11 and the control ECU 12.
[0147] Furthermore, as another embodiment relating to the first embodiment, a traffic flow matching trajectory generation model for generating a traffic flow matching trajectory T1 and a setting matching trajectory generation model for generating a setting matching trajectory T2 may be provided, and the traffic flow matching trajectory T1 and the setting matching trajectory T2 may be generated by separate trajectory generation models. In this case, traffic flow data does not need to be input to the setting matching trajectory model.
[0148] Furthermore, in another embodiment, the experience judgment unit F42 may be implemented on an AI basis. For example, the experience judgment unit F42 may use the information accumulated in the experience DB 13c as training data to train an experience model constructed based on a neural network. Then, information regarding the scene or scenario that vehicle 1 is currently encountering may be input to the experience model, and information indicating whether or not speeding is permissible may be output in response to this input.
[0149] In another embodiment, the traffic flow data may consist only of relatively low-precision data at the node level. In this case, the traffic flow-matched trajectory T1 may be a trajectory that causes vehicle 1 to travel at a speed matched to the speed of the traffic flow at the node level.
[0150] In another embodiment, if the user's intention cannot be confirmed due to the absence of input from the user's operation input device 25 in response to the presented content, the user may assume that the current intention is to prohibit speeding. On the other hand, if the user's intention cannot be confirmed, the user may assume that the current intention is to permit speeding.
[0151] Furthermore, in other embodiments, the vehicle 1 equipped with the driving system DS may be a right-hand drive vehicle or a left-hand drive vehicle. In addition, the traffic environment in which the vehicle 1 travels may be a traffic environment based on left-hand traffic or a traffic environment based on right-hand traffic. The driving system DS according to this disclosure may be appropriately optimized in consideration of the road traffic laws and customs of each country and region, as well as the forms of police investigations, prosecutors, criminal proceedings, or civil proceedings for traffic accidents.
[0152] In another embodiment, the computer constituting the driving system DS may incorporate circuits such as FPGAs. Then, a portion of the processing performed by the driving system DS may be realized by the processor controlling the circuits based on a computer program, or by the circuits operating independently.
[0153] (Disclosure of Technical Ideas) This specification discloses several technical ideas as described in the following paragraphs. Some paragraphs are written in a multiple dependent form, where subsequent paragraphs alternately refer to preceding paragraphs. These paragraphs written in a multiple dependent form define several technical ideas.
[0154] <Technical Concept 1> A vehicle system comprising at least one processing unit (10a, 11a, 12a, 13a) that performs processing for autonomously driving a vehicle (1), wherein the at least one processing unit is configured to acquire data indicating the external environment of the vehicle, and to grasp the speed of the traffic flow based on the data and generate a planned trajectory (T1) for driving the vehicle at a speed matching the speed of the traffic flow.
[0155] <Technical Concept 2> The vehicle system according to Technical Concept 1, wherein the traffic flow is calculated based on the speed of agents present around the vehicle recognized based on the data, and is expressed in a representation method that includes speeds expressed for each predetermined unit that divides the road area.
[0156] <Technical Concept 3> The vehicle system according to Technical Concept 1, wherein the traffic flow includes both a narrow-area traffic flow (NTF) that represents the traffic flow in a narrow area around the vehicle recognized based on the data with high resolution, and a wide-area traffic flow (WTF) distributed from a traffic flow server (90) that represents the traffic flow in a wider area than the narrow-area traffic flow with lower resolution than the narrow-area traffic flow.
[0157] <Technical Concept 4> The vehicle system according to any one of Technical Concepts 1 to 3, wherein the at least one processing unit generates a plurality of planned trajectories (T2) including a planned trajectory that causes the vehicle to travel at a speed matching the speed of the traffic flow and a planned trajectory that causes the vehicle to travel at a speed different from the speed of the traffic flow.
[0158] <Technical Concept 5> The vehicle system according to Technical Concept 4, wherein the planned trajectory for driving the vehicle at a speed different from the speed of the traffic flow is a planned trajectory for driving the vehicle at a speed based on a speed setting predetermined by the vehicle user.
[0159] <Technical Concept 6> The vehicle system according to Technical Concept 4 or 5, wherein the at least one processing unit determines, based on information provided from outside the vehicle system regarding the permissibility of exceeding the speed limit, whether or not to adopt the planned trajectory for driving the vehicle at a speed matched to the speed of the traffic flow as the trajectory used for controlling the vehicle, if the planned trajectory for driving the vehicle at a speed matched to the speed of the traffic flow exceeds the speed limit.
[0160] <Technical Concept 7> The vehicle system according to any one of Technical Concepts 1 to 6, wherein the at least one processing unit is configured to further perform: confirm the user's intention regarding exceeding the speed limit; prohibit the adoption of the planned trajectory that exceeds the speed limit if the user's intention indicates prohibition of exceeding the speed limit; and allow the adoption of the planned trajectory that exceeds the speed limit if the user's intention indicates permission to exceed the speed limit.
[0161] <Technical Idea 8> A vehicle system according to technical idea 1 or 7, further comprising situation information representing the situation the vehicle is encountering when the user of the vehicle expresses an intention to exceed the speed limit, and an experience database (13c) that stores the result of the user's judgment at that time, wherein the at least one processing unit is configured to further perform: identify the situation the vehicle is currently encountering; identify, based on the information in the experience database, what kind of intention the user has expressed in the past in situations that match or are similar to the identified situation; and, if the user's past intention indicates prohibiting exceeding the speed limit, prohibit the adoption of the planned trajectory that exceeds the speed limit, and if the user's past intention indicates permitting exceeding the speed limit, permit the adoption of the planned trajectory that exceeds the speed limit.
[0162] <Technical Concept 9> The vehicle system according to any one of Technical Concepts 1 to 8, wherein the at least one processing unit is configured to generate a plurality of planned trajectories, and to further select one trajectory from the plurality of planned trajectories to be adopted as a trajectory to be used for controlling the vehicle, and at least one of the understanding of the speed of the traffic flow and the generation of the planned trajectories is realized based on processing of an artificial intelligence model using a neural network, and the selection of the trajectory is realized based on rule-based processing.
[0163] <Technical Concept 10> The vehicle system according to Technical Concept 9, wherein the determination of the speed of the traffic flow is realized based on the processing of the first model as an artificial intelligence model, the generation of the planned trajectory is realized based on the processing of the second model as an artificial intelligence model, and the traffic flow data representing the traffic flow, output from the first model and input to the second model, is passed between the first model and the second model in the form of high-dimensional data containing a large number of features.
Claims
1. A vehicle system comprising at least one processing unit (10a, 11a, 12a, 13a) that performs processing for autonomously driving a vehicle (1), wherein the at least one processing unit is configured to acquire data indicating the external environment of the vehicle, and to grasp the speed of the traffic flow based on the data and generate a planned trajectory (T1) for driving the vehicle at a speed matching the speed of the traffic flow.
2. The vehicle system according to claim 1, wherein the traffic flow is calculated based on the speeds of agents present around the vehicle recognized based on the data, and is expressed in a representation that includes speeds expressed for each predetermined unit that divides the road area.
3. The vehicle system according to claim 1, wherein the traffic flow includes both a narrow-area traffic flow (NTF) that represents the traffic flow in a narrow area around the vehicle recognized based on the data with high resolution, and a wide-area traffic flow (WTF) distributed from a traffic flow server (90) that represents the traffic flow in a wider area than the narrow-area traffic flow with lower resolution than the narrow-area traffic flow.
4. The vehicle system according to claim 1, wherein the at least one processing unit generates a plurality of planned trajectories (T2) including a planned trajectory that causes the vehicle to travel at a speed matching the speed of the traffic flow and a planned trajectory that causes the vehicle to travel at a speed different from the speed of the traffic flow.
5. The vehicle system according to claim 4, wherein the planned trajectory for driving the vehicle at a speed different from the speed of the traffic flow is a planned trajectory for driving the vehicle at a speed based on a speed setting predetermined by the vehicle user.
6. The vehicle system according to claim 4 or 5, wherein the at least one processing unit determines, based on information provided from outside the vehicle system regarding the permissibility of exceeding the speed limit, whether or not to adopt the planned trajectory for driving the vehicle at a speed matched to the speed of the traffic flow as the trajectory used for controlling the vehicle, if the planned trajectory for driving the vehicle at a speed matched to the speed of the traffic flow exceeds the speed limit.
7. The vehicle system according to claim 1, wherein the at least one processing unit is configured to further perform: confirm the user's intention regarding exceeding the speed limit; prohibit the adoption of the planned trajectory that exceeds the speed limit if the user's intention indicates prohibition of exceeding the speed limit; and allow the adoption of the planned trajectory that exceeds the speed limit if the user's intention indicates permission to exceed the speed limit.
8. The vehicle system according to claim 1 or 7, further comprising situation information representing the situation the vehicle is encountering when the user of the vehicle expresses an intention to exceed the speed limit, and an experience database (13c) that stores the result of the user's judgment at that time, wherein the at least one processing unit is configured to further perform: identify the situation the vehicle is currently encountering; identify, based on the information in the experience database, what kind of intention the user has expressed in the past in situations that match or are similar to the identified situation; and, if the user's past intention indicates prohibiting exceeding the speed limit, prohibit the adoption of the planned trajectory that exceeds the speed limit, and if the user's past intention indicates allowing exceeding the speed limit, allow the adoption of the planned trajectory that exceeds the speed limit.
9. The vehicle system according to claim 1, wherein the at least one processing unit is configured to generate a plurality of planned trajectories, and to further select one trajectory from the plurality of planned trajectories to be used as a trajectory for controlling the vehicle, and at least one of the understanding of the speed of the traffic flow and the generation of the planned trajectories is realized based on processing of an artificial intelligence model using a neural network, and the selection of the trajectory is realized based on rule-based processing.
10. The vehicle system according to claim 9, wherein the determination of the speed of the traffic flow is realized based on processing by the first model as the artificial intelligence model, the generation of the planned trajectory is realized based on processing by the second model as the artificial intelligence model, and the traffic flow data representing the traffic flow, output from the first model and input to the second model, is passed between the first model and the second model in the form of high-dimensional data containing a large number of features.
11. A vehicle system comprising at least one processing unit (10a, 11a, 12a, 13a) that performs processing for autonomously driving a vehicle, wherein the at least one processing unit is configured to: confirm the user's intention regarding exceeding the speed limit; generate one or more planned trajectories (T1, T2) for driving the vehicle; and, if the user's intention indicates prohibiting exceeding the speed limit, prohibit the adoption of the planned trajectory that exceeds the speed limit, and if the user's intention indicates allowing exceeding the speed limit, allow the adoption of the planned trajectory that exceeds the speed limit.
